基于DDPG算法的V2G用户特征识别及定价策略

doi:10.3969/j.issn.2097-0706.2026.07.007

摘要/Abstract

摘要：

分布式光伏等可再生能源的大规模并网以及电动汽车数量的快速增长导致电力系统运行面临的不确定性问题日趋明显。随着电动汽车入网（V2G）技术的逐步发展和推广，计及V2G策略的电动汽车充放电站运行优化逐步受到关注。然而电动汽车用户群体的充放电行为是大量个体用户行为特征的聚合，常难以准确获知用户的充放电偏好并据此优化制定V2G回购电价。针对此问题，对电动汽车充放电站与分布式光伏、区域负荷聚合的虚拟电厂（VPP），构建了一种考虑车网互动的优化调度模型，以VPP综合效益最大化为目标，计及电动汽车充放电功率与新能源发电出力等约束。考虑到大量电动汽车用户个体的充放电行为及分布式光伏发电的出力特性难以精确建模，采用基于强化学习的交互式优化调度方法进行求解，并对比分析了若干常用深度强化学习算法的求解性能。进一步，提出了一种电动汽车用户行为特征的识别方法，并在此基础上建立了充放电站运营商对用户V2G放电的定价策略。采用放电荷电阈值与放电差价阈值两个参数，分别描述电动汽车用户对续航里程的焦虑程度和对放电收益的经济性偏好，并提出了辨识这些阈值的系统化流程。将辨识所得阈值引入VPP优化调度模型，提出了电动汽车充放电站中V2G放电电价的定价策略。基于深度确定性策略梯度（DDPG）算法，通过若干典型场景下的仿真算例对电动汽车用户充放电行为的荷电阈值、差价阈值等特征参数进行识别，进而优化充放电站的V2G电价。算例结果表明，电动汽车用户行为的特征差异对VPP总效益具有显著影响，所提出V2G定价策略能更好地兼顾电动汽车用户的响应意愿与系统经济性，显著提升VPP的整体运行效益。

关键词: 电动汽车入网, 分布式能源, 虚拟电厂, 定价策略, 强化学习

Abstract:

The large-scale grid integration of renewable energy sources such as distributed photovoltaic （PV） generation and the rapid growth in the number of electric vehicles （EV） have led to increasingly prominent uncertainties in the operation of power systems. With the gradual development and promotion of vehicle-to-grid （V2G） technology， the optimization of EV charging and discharging station operation considering V2G strategies has gradually attracted increasing attention. However， the charging and discharging behaviors of EV users represent the aggregation of the behavioral characteristics of a large number of individual users， making it difficult to accurately identify users' charging and discharging preferences and optimize V2G electricity purchase prices accordingly. To address this issue， an optimization scheduling model considering vehicle-grid interaction was constructed for a virtual power plant （VPP） consisting of EV charging and discharging stations， distributed PV generation， and aggregated regional loads. This model aims to maximize the comprehensive benefits of the VPP while considering constraints such as EV charging and discharging power and renewable energy generation output. Considering the difficulty in accurately modeling the charging and discharging behaviors of individual EV users and the output characteristics of distributed PV generation， an interactive optimization scheduling method based on reinforcement learning was employed for solving the problem， and the solution performance of several commonly used deep reinforcement learning algorithms was compared and analyzed. Furthermore， a method for identifying EV user behavioral characteristics was proposed， and a pricing strategy for EV charging and discharging station operators regarding users' V2G discharging was developed. Two parameters， namely the discharge state-of-charge threshold and the discharge price difference threshold， were used to respectively characterize EV users' range anxiety and economic preference for discharge benefits， and a systematic procedure for identifying these thresholds was proposed. The identified thresholds were incorporated into the VPP optimization scheduling model， and a pricing strategy for V2G discharge electricity prices at EV charging and discharging stations was proposed. Based on the deep deterministic policy gradient （DDPG） algorithm， the characteristic parameters of EV users' charging and discharging behaviors， including the state-of-charge threshold and discharge price difference threshold， were identified through simulation cases under several typical scenarios， thereby optimizing V2G electricity prices at charging and discharging stations. The results of the case studies indicated that differences in EV users' behavioral characteristics significantly affected the overall benefits of the VPP. The proposed V2G pricing strategy better balanced EV users' willingness to respond and system economic efficiency， thereby significantly improving the overall operational benefits of the VPP.

Key words: vehicle to grid, distributed energy, virtual power plant, pricing strategy, reinforcement learning

中图分类号:

TK01：TM73

李明扬, 包金龙. 基于DDPG算法的V2G用户特征识别及定价策略[J]. 综合智慧能源, 2026, 48(7): 68-73.

LI Mingyang, BAO Jinlong. V2G user feature recognition and pricing strategy using DDPG algorithm[J]. Integrated Intelligent Energy, 2026, 48(7): 68-73.

图/表 12

图1

图2

图3

图4

表1

表2

表3

图5

表4

图6

表5

表6

参考文献 26

[1]	马苗苗, 任智伟, 刘立成, 等. 考虑新能源消纳的电动汽车有序充电控制策略[J]. 太阳能学报, 2024, 45(8): 94-103.
	MA Miaomiao, REN Zhiwei, LIU Licheng, et al. Orderly charging control strategy for electric vehicles considering new energy accommodation[J]. Acta Energiae Solaris Sinica, 2024, 45(8): 94-103.
[2]	张忠静, 刘磊, 蔡永翔, 等. 新型电力系统下大规模新能源接入配电网的线损变化趋势分析[J]. 电力大数据, 2024, 27(2): 11-22.
	ZHANG Zhongjing, LIU Lei, CAI Yongxiang, et al. Analysis of line loss change trend of large-scale new energy access distribution network under new power system[J]. Power Systems and Big Data, 2024, 27(2): 11-22.
[3]	刘红. V2G模式下“车-站-网”融合的电动汽车充放电调度研究[D]. 西安:西安理工大学, 2024.
	LIU Hong. Research on charging and discharging scheduling of electric vehicle based on "vehicle-station-network" integration in V2G mode[D]. Xi'an: Xi'an University of Technology, 2024.
[4]	LYU Y, DING Y Y, DIAO R H, et al. Master-slave game optimization scheduling of virtual power plant considering electric vehicles[C]// Proceedings of 2023 3rd International Conference on Intelligent Power and Systems (ICIPS). IEEE, 2024: 728-731.
[5]	程明, 张新燕, 王志磊, 等. 考虑接入电动汽车充电站的虚拟电厂低碳经济调度[J]. 科学技术与工程, 2024, 24(19): 8115-8126.
	CHENG Ming, ZHANG Xinyan, WANG Zhilei, et al. Low carbon economic dispatch of virtual power plant considering access to electric vehicle charging station[J]. Science Technology and Engineering, 2024, 24(19): 8115-8126.
[6]	潘振华, 孙丽江. 含电动汽车的虚拟电厂经济优化调度分析[J]. 上海电机学院学报, 2024, 27(3): 131-136.
	PAN Zhenhua, SUN Lijiang. Economic optimization scheduling analysis of virtual power plant including electric vehicles[J]. Journal of Shanghai Dianji University, 2024, 27(3): 131-136.
[7]	LI Z H, BAO Y S. Joint dispatching strategy of distributed generation and electric vehicle based on virtual power plant[C]// Proceedings of 2020 IEEE 4th Conference on Energy Internet and Energy System Integration (EI2). IEEE, 2021: 1313-1318.
[8]	薛东. 基于粒子群算法的聚合电动汽车虚拟电厂经济性分析[J]. 安徽电气工程职业技术学院学报, 2021, 26(4): 36-45.
	XUE Dong. Economic analysis of electric vehicle virtual power plant based on particle swarm algorithm[J]. Journal of Anhui Electrical Engineering Professional Technique College, 2021, 26(4): 36-45.
[9]	钱科军, 顾水福, 杨永标. 含电动汽车的虚拟电厂充放电多时间尺度调控技术[J]. 电网与清洁能源, 2025, 41(11): 81-86.
	QIAN Kejun, GU Shuifu, YANG Yongbiao. Multi-time scale control technology for charging and discharging in virtual power plants containing electric vehicles[J]. Power System and Clean Energy, 2025, 41(11): 81-86.
[10]	LIU K J, XIA Y H, BO Y L, et al. Economic scheduling strategy of virtual power plant with electric vehicles[C]// Proceedings of 2023 IEEE 7th Conference on Energy Internet and Energy System Integration (EI2). IEEE, 2024: 2327-2332.
[11]	杨娜. 基于主从博弈的含电动汽车的虚拟电厂经济优化运行研究[D]. 银川: 宁夏大学, 2025.
	YANG Na. Research on economic optimal operation of virtual power plant with electric vehicles based on master-slave game[D]. Yinchuan: Ningxia University, 2025.
[12]	徐胜涛. 计及规模化电动汽车的虚拟电厂聚合及优化运行研究[D]. 南京: 南京邮电大学, 2024.
	XU Shengtao. Research on virtual power plant aggregation and optimal operation considering large-scale electric vehicles[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2024.
[13]	GAO J, ZHANG L H. A power dispatching optimization method of virtual power plants considering electric vehicle load aggregation[C]// Proceedings of 2025 2nd International Conference on Electrical Technology and Automation Engineering (ETAE). IEEE, 2025: 60-63.
[14]	李明扬, 张智. 基于强化学习的含分布式风-光-储虚拟电厂优化调度[J]. 智慧电力, 2024, 52(8): 50-56.
	LI Mingyang, ZHANG Zhi. Optimal dispatch of distributed wind-solar-storage virtual power plants based on reinforcement learning[J]. Smart Power, 2024, 52(8): 50-56.
[15]	孙冬川, 孙亮, 孔令乾, 等. 基于深度强化学习的多能虚拟电厂优化调度[J]. 东北电力大学学报, 2024, 44(3): 102-111.
	SUN Dongchuan, SUN Liang, KONG Lingqian, et al. Deep reinforcement learning-based optimal scheduling for multi-energy virtual power plant[J]. Journal of Northeast Dianli University (Natural Science Edition), 2024, 44(3): 102-111.
[16]	TANG Y Y, LV R, XU Z F, et al. A virtual power plant economic scheduling strategy based on deep reinforcement learning[C]// Proceedings of 2023 5th International Conference on Power and Energy Technology (ICPET). IEEE, 2023: 1365-1370.
[17]	CHAO F A Z, YING S B Z, YU T C J. Application of improved reinforcement learning technology for real time operation and scheduling optimization of virtual power plant[C]// Proceedings of 2023 IEEE Sustainable Power and Energy Conference (iSPEC). IEEE, 2024: 1-6.
[18]	MA H Y, LI G D, ZHANG Y D, et al. Research on optimal scheduling strategy of virtual power plant multi-operation scenarios based on deep reinforcement learning[C]// Proceedings of 2024 Second International Conference on Cyber-Energy Systems and Intelligent Energy (ICCSIE). IEEE, 2024: 1-6.
[19]	ZHANG S L, YE F, ZHOU J X, et al. Efficient virtual power plant dispatch model for distributed energy resources using deep reinforcement learning[C]// Proceedings of 2023 IEEE 13th International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER). IEEE, 2023: 510-514.
[20]	袁孝科, 沈石兰, 张茂松, 等. 基于可解释强化学习的智能虚拟电厂最优调度[J]. 综合智慧能源, 2025, 47(1): 1-9. doi: 10.3969/j.issn.2097-0706.2025.01.001
	YUAN Xiaoke, SHEN Shilan, ZHANG Maosong, et al. Optimal scheduling of intelligent virtual power plants based on explainable reinforcement learning[J]. Integrated Intelligent Energy, 2025, 47(1): 1-9. doi: 10.3969/j.issn.2097-0706.2025.01.001
[21]	李明扬, 窦梦园. 基于强化学习的含电动汽车虚拟电厂优化调度[J]. 综合智慧能源, 2024, 46(6): 27-34. doi: 10.3969/j.issn.2097-0706.2024.06.004
	LI Mingyang, DOU Mengyuan. Optimal scheduling of virtual power plants integrating electric vehicles based on reinforcement learning[J]. Integrated Intelligent Energy, 2024, 46(6): 27-34. doi: 10.3969/j.issn.2097-0706.2024.06.004
[22]	王琦, 杨毅远, 江季. EASY RL强化学习教程[M]. 北京: 人民邮电出版社, 2022.
[23]	刘文军, 李帅虎, 马瑞, 等. 低压配电台区柔性互联系统多目标DDPG优化调度[J]. 智慧电力, 2024, 52(6): 62-70.
	LIU Wenjun, LI Shuaihu, MA Rui, et al. Multi-objective DDPG optimal dispatch for low-voltage distribution station area flexible interconnection system[J]. Smart Power, 2024, 52(6): 62-70.
[24]	PANGGABEAN J, SUTISNA N, SYAFALNI I, et al. Comparison of MPPT based on deep reinforcement learning by DQN, DDPG and TD3[C]// Proceedings of 2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 2023: 261-266.
[25]	SANG H Y, WANG S Q. Motion planning of space robot obstacle avoidance based on DDPG algorithm[C]// Proceedings of 2022 International Conference on Service Robotics (ICoSR). IEEE, 2023: 175-181.
[26]	郑庆明, 井延伟, 梁涛, 等. 基于DDPG算法的离网型可再生能源大规模制氢系统优化调度[J]. 综合智慧能源, 2024, 46(6): 35-43. doi: 10.3969/j.issn.2097-0706.2024.06.005
	ZHENG Qingming, JING Yanwei, LIANG Tao, et al. Optimized scheduling on large-scale hydrogen production system for off-grid renewable energy based on DDPG algorithm[J]. Integrated Intelligent Energy, 2024, 46(6): 35-43. doi: 10.3969/j.issn.2097-0706.2024.06.005

时段	电网电费	服务费	EV充电	EV放电
00:00 — 06:00	0.40	0.24	0.64	0.40
06:00 —12:00	1.00	0.18	1.18	1.00
12:00 —14:00	0.40	0.24	0.64	0.40
14:00 —16:00	1.00	0.18	1.18	1.00
16:00 —24:00	1.60	0.13	1.73	1.20

季节	时段	电网电费	服务费	EV充电	EV放电
夏	20:00 —24:00	1.92	0.11	2.03	1.40
冬	18:00 —22:00	1.92	0.11	2.03	1.40

算法	运行时间/min	奖励值/元
DDPG	55.41	-1 102
TD3	90.36	-1 092
SAC	98.58	-1 162

$\widehat{\mathit{\lambda }}$/[元·(kW·h)^-1]	$\widehat{\mathit{Q}}$
	春				夏				秋				冬
	0.40	0.55	0.70	0.85	0.40	0.55	0.70	0.85	0.40	0.55	0.70	0.85	0.40	0.55	0.70	0.85
0.1	-794	-965	-1 109	-1 557	-44	-118	-302	-870	-1 017	-1 292	-1 545	-1 871	-605	-712	-1 032	-1 225
0.3	-757	-1 033	-1 189	-1 477	-77	-118	-337	-807	-1 082	-1 308	-1 559	-1 732	-459	-596	-1 107	-1 280
0.5	-811	-1 009	-1 111	-1 512	-16	-127	-271	-740	-1 111	-1 344	-1 597	-1 790	-457	-678	-1 135	-1 232
0.7	-2 845	-2 826	-2 819	-2 830	-15	-102	-224	-909	-3 225	-3 229	-3 194	-3 217	-640	-735	-1 037	-1 242

场景	${\mathit{C}}^{\mathrm{p}\mathrm{r}\mathrm{o}}$	${\mathit{C}}^{\mathrm{l}\mathrm{o}\mathrm{a}\mathrm{d}\_\mathrm{g}\mathrm{r}\mathrm{i}\mathrm{d}}$	${\mathit{C}}_{\mathrm{e}\mathrm{v}}^{\mathrm{s}\mathrm{a}\mathrm{t}}$	总效益
1	1 500	1 875	642	-1 017
2	1 501	1 872	710	-1 082
3	1 492	1 904	699	-1 111
4	867	4 042	50	-3 225
5	1 397	2 409	280	-1 292
6	1 404	2 326	385	-1 308
7	1 387	2 423	309	-1 344
8	877	4 090	16	-3 229
9	1 320	2 709	156	-1 545
10	1 332	2 617	275	-1 559
11	1 320	2 673	245	-1 597
12	878	4 046	25	-3 194
13	1 237	2 922	186	-1 871
14	1 281	2 776	238	-1 732
15	1 272	2 810	252	-1 790
16	871	4 069	20	-3 217