关键词: 虚拟电厂, 电动汽车, V2G, 分布式新能源, 深度确定性策略梯度算法, 优化调度, 强化学习

Abstract:

Disorderly charging behaviors of massive electric vehicles （EVs） may cause violent fluctuations in power loads， affecting the security and stability of the power grid. With the application of vehicle to grid（V2G） technology， the scheduling method can be optimized by aggregating EV charging stations and surrounded distributed renewable energy generators into a virtual power plant（VPP）. The aggregation can improve the economy of charging behaviors and satisfaction of EV users， raise the utilization rate of distributed renewable energy， and mitigate load fluctuations in the grid. However， the overall charging or discharging load is the aggregation result of random charging or discharging behaviors of massive individual EVs， which is difficult to be accurately described by mathematical models. Thus，an interactive optimal scheduling framework based on deep reinforcement learning is presented for a VPP including EVs， with the objective of maximizing the benefit of all EV users in the VPP. The VPP control center，serving as an intelligent agent， can decide the charging and discharging of individual EVs without their detailed models. The agent continuously learns and updates its strategies through interactions with regional grids， overcoming the limitations of centralized optimal scheduling. The framework is solved by Deep Deterministic Policy Gradient（DDPG） algorithm. Simulation results show that， compared with the centralized scheduling， the proposed method improves the benefits of individual EV users， and the coordinative scheduling of EV charging/discharging loads and renewable energy outputs shaves the peak loads in the grid， and boosts the overall performance of the VPP.

Key words: virtual power plant, electric vehicle, vehicle to grid, distributed renewable energy, DDPG algorithm, optimal scheduling, reinforcement learning