基于可解释强化学习的智能虚拟电厂最优调度

doi:10.3969/j.issn.2097-0706.2025.01.001

摘要/Abstract

摘要：

随着电动汽车的不断普及，能源系统日益复杂。虚拟电厂（VPP）可以通过物联网和人工智能技术，将分布式电源、储能系统、可控负荷以及EV等分布式能源进行聚合和协调优化，有助于提升能源的使用效率，并促进非可再生能源的消纳，增强电网稳定性。现阶段人工智能技术在电力系统等安全要求较高的应用领域缺乏可靠性和透明度，可能导致用户和运营商难以理解算法如何做出特定的能源调配决策。针对人工智能技术下的VPP实现最优调度并兼顾解释其决策过程的平衡问题，提出一种可解释强化学习的交互式框架，使用近端策略优化算法实现VPP的最优调度，并使用决策树建立一种可解释性强化学习框架，用于提供透明的决策支持，使非专业用户能够理解人工智能在调节能源系统方面的决策过程。试验表明，与传统强化学习优化方法相比，该方法不仅提高了能源分配的效率，而且通过增强模型的可解释性，加强了用户对智能VPP管理系统的信任。

关键词: 虚拟电厂, 电动汽车, 近端策略优化算法, 强化学习, 决策树, 可解释性框架, 分布式电源, 人工智能

Abstract:

With the increasing popularity of electric vehicles（EVs），energy systems are becoming more complex. Virtual power plants（VPPs）can aggregate and optimize distributed energy resources such as distributed generation，energy storage systems，controllable loads，and EVs through internet of things（IoT）and artificial intelligence（AI）technologies，enhancing energy efficiency and facilitating the consumption of non-renewable energy while reinforcing grid stability. However，current AI technologies lack reliability and transparency in high-safety applications like power systems，potentially making it challenging for users and operators to understand how algorithms make specific energy allocation decisions. To address the balance between achieving optimal scheduling of VPPs utilizing AI and explaining the decision-making processes，this study proposed an interactive framework based on explainable reinforcement learning. This framework employed the proximal policy optimization（PPO）algorithm for optimal scheduling of VPPs and constructed an explainable reinforcement learning framework using decision trees to provide transparent decision support that enabled non-expert users to understand AI's decision-making processes in regulating energy systems. The results indicated that compared to traditional reinforcement learning optimization methods，this approach not only improved energy allocation efficiency but also strengthened user trust in intelligent VPP management systems by enhancing model interpretability.

Key words: virtual power plant, electric vehicle, proximal policy optimization algorithm, reinforcement learning, decision tree, explainable framework, distributed energy, artificial intelligence

中图分类号:

TK019

袁孝科, 沈石兰, 张茂松, 石晨旭, 杨凌霄. 基于可解释强化学习的智能虚拟电厂最优调度[J]. 综合智慧能源, 2025, 47(1): 1-9.

YUAN Xiaoke, SHEN Shilan, ZHANG Maosong, SHI Chenxu, YANG Lingxiao. Optimal scheduling of intelligent virtual power plants based on explainable reinforcement learning[J]. Integrated Intelligent Energy, 2025, 47(1): 1-9.

图/表 9

图1

图2

图3

图4

图5

图6

图7

图8

图9

参考文献 26

[1]	刘媛媛, 刘芳芳, 贾天翔, 等. 商住园区综合能源供暖(冷)系统的方案设计及运行经济性研究[J]. 综合智慧能源, 2023, 45(12): 20-28. doi: 10.3969/j.issn.2097-0706.2023.12.003
	LIU Yuanyuan, LIU Fangfang, JIA Tianxiang, et al. Design of the integrated energy heating (cooling) system for a commercial and residential park and its economy analysis[J]. Integrated Intelligent Energy, 2023, 45(12): 20-28. doi: 10.3969/j.issn.2097-0706.2023.12.003
[2]	田泽禹, 沙钊旸, 赵全斌, 等. 针对温控负载变化的虚拟电厂控制策略研究[J]. 综合智慧能源, 2024, 46(1): 28-37. doi: 10.3969/j.issn.2097-0706.2024.01.004
	TIAN Zeyu, SHA Zhaoyang, ZHAO Quanbin, et al. Research on control strategy for virtual power plants in response to thermostatically controlled loads[J]. Integrated Intelligent Energy, 2024, 46(1): 28-37. doi: 10.3969/j.issn.2097-0706.2024.01.004
[3]	孙秋野, 姚葭, 王一帆. 从虚拟电厂到真实电量: 虚拟电厂研究综述与展望[J]. 发电技术, 2023, 4(5): 583-601.
	SUN Qiuye, YAO Jia, WANG Yifan. From virtual power plant to real electricity:Summary and prospect of virtual power plant research[J]. Power Generation Technology, 2023, 44(5): 583-601. doi: 10.12096/j.2096-4528.pgt.23102
[4]	葛晓琳, 郝广东, 夏澍, 等. 考虑规模化电动汽车与风电接入的随机解耦协同调度[J]. 电力系统自动化, 2020, 44(4): 54-62.
	GE Xiaolin, HAO Guangdong, XIA Shu, et al. Stochastic decoupling collaborative dispatch considering integration of large-scale electric vehicles and wind power[J]. Automation of Electric Power Systems, 2020, 44(4): 54-62.
[5]	冶兆年, 赵长禄, 王永真, 等. 基于纳什议价的共享储能能源互联网络双目标优化[J]. 综合智慧能源, 2022, 44(7):40-48. doi: 10.3969/j.issn.2097-0706.2022.07.005
	YE Zhaonian, ZHAO Changlu, WANG Yongzhen, et al. Dua-objective optimization of energy networks with shared energy storage based on Nash bargaining[J]. Integrated Intelligent Energy, 2022, 44(7): 40-48. doi: 10.3969/j.issn.2097-0706.2022.07.005
[6]	刘建行, 刘方. 基于深度强化学习的梯级水蓄风光互补系统优化调度策略研究[J]. 广东电力, 2024, 37(5): 10-22.
	LIU Jianhang, LIU Fang. Research on optimized dispatching strategy of cascade hydropower-pumping-storage-wind-photovoltaic multi-energy complementary system based on deep reinforcement learning[J]. Guangdong Electric Power, 2024, 37(5): 10-22.
[7]	林彦旭, 高辉. 基于SSA-VMD-BiLSTM模型的充电站负荷预测方法[J]. 广东电力, 2024, 37(6): 53-61.
	LIN Yanxu, GAO Hui. Load prediction method of charging station based on SSA-VMD-BiLSTM model[J]. Guangdong Electric Power, 2024, 37(6): 53-61.
[8]	DA SILVA F L, NISHIDA C E H, ROIJERS D M, et al. Coordination of electric vehicle charging through multiagent reinforcement learning[J]. IEEE Transactions on Smart Grid, 2020, 11(3): 2347-2356.
[9]	PAN X T, WANG L P, QIU Q C, et al. Many-objective optimization for large-scale EVs charging and discharging schedules considering travel convenience[J]. Applied Intelligence, 2022, 52(3): 2599-2620.
[10]	ZHANG F Y, YANG Q Y, AN D. CDDPG:A deep-reinforcement-learning-based approach for electric vehicle charging control[J]. IEEE Internet of Things Journal, 2021, 8(5): 3075-3087.
[11]	LI H, LI G J, LIE T T, et al. Constrained large-scale real-time EV scheduling based on recurrent deep reinforcement learning[J]. International Journal of Electrical Power & Energy Systems, 2023, 144: 108603.
[12]	杨挺, 赵黎媛, 刘亚闯, 等. 基于深度强化学习的综合能源系统动态经济调度[J]. 电力系统自动化, 2021, 45(5): 39-47.
	YANG Ting, ZHAO Liyuan, LIU Yachuang, et al. Dynamic economic dispatch for integrated energy system based on deep reinforcement learning[J]. Automation of Electric Power Systems, 2021, 45(5): 39-47.
[13]	杨志学, 任洲洋, 孙志媛, 等. 基于近端策略优化算法的新能源电力系统安全约束经济调度方法[J]. 电网技术, 2023, 47(3): 988-998.
	YANG Zhixue, REN Zhouyang, SUN Zhiyuan, et al. Security-constrained economic dispatch of renewable energy integrated power systems based on proximal policy optimization algorithm[J]. Power System Technology, 2023, 47(3): 988-998.
[14]	WAN Z Q, LI H P, HE H B, et al. Model-free real-time EV charging scheduling based on deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2019, 10(5): 5246-5257.
[15]	LI H P, WAN Z Q, HE H B. Constrained EV charging scheduling based on safe deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2020, 11(3): 2427-2439.
[16]	黎静华, 朱梦姝, 陆悦江, 等. 综合能源系统优化调度综述[J]. 电网技术, 2021, 45(6): 2256-2272.
	LI Jinghua, ZHU Mengshu, LU Yuejiang, et al. Review on optimal scheduling of integrated energy systems[J]. Power System Technology, 2021, 45(6): 2256-2272.
[17]	蔺伟山, 王小君, 孙庆凯, 等. 计及安全约束的综合能源系统深度强化学习优化调度策略研究[J]. 电网技术, 2023, 47(5): 1970-1983.
	LIN Weishan, WANG Xiaojun, SUN Qingkai, et al. Optimal dispatch strategy of integrated energy system based on deep reinforcement learning considering security constraints[J]. Power System Technology, 2023, 47(5): 1970-1983.
[18]	李佳玮. 基于图神经网络的配电网故障定位方法[D]. 北京: 北京交通大学, 2022.
	LI Jiawei. Fault location method of distribution network based on graph neural network[D]. Beijing: Beijing Jiaotong University, 2022.
[19]	崔杨, 王议坚, 黄彦浩, 等. 基于多元注意力框架与引导式监督学习的闭环风电功率超短期预测策略[J]. 中国电机工程学报, 2023, 43(4): 1334-1347.
	CUI Yang, WANG Yijian, HUANG Yanhao, et al. Closed-loop wind power ultra-short-term forecasting strategy based on multi-attention framework and guided supervised learning[J]. Proceedings of the CSEE, 2023, 43(4): 1334-1347.
[20]	HEO S, KO J, KIM S, et al. Explainable AI-driven net-zero carbon roadmap for petrochemical industry considering stochastic scenarios of remotely sensed offshore wind energy[J]. Journal of Cleaner Production, 2022, 379: 134793.
[21]	孙庆凯, 王小君, 张义志, 等. 基于LSTM和多任务学习的综合能源系统多元负荷预测[J]. 电力系统自动化, 2021, 45(5): 63-70.
	SUN Qingkai, WANG Xiaojun, ZHANG Yizhi, et al. Multiple load prediction of integrated energy system based on long short-term memory and multi-task learning[J]. Automation of Electric Power Systems, 2021, 45(5): 63-70.
[22]	周挺, 杨军, 詹祥澎, 等. 一种数据驱动的暂态电压稳定评估方法及其可解释性研究[J]. 电网技术, 2021, 45(11): 4416-4425.
	ZHOU Ting, YANG Jun, ZHAN Xiangpeng, et al. Data-driven method and interpretability analysis for transient voltage stability assessment[J]. Power System Technology, 2021, 45(11): 4416-4425.
[23]	龙寰, 杨婷, 徐劭辉, 等. 基于数据驱动的风电机组状态监测与故障诊断技术综述[J]. 电力系统自动化, 2023, 47(23): 55-69.
	LONG Huan, YANG Ting, XU Shaohui, et al. Review of data-driven condition monitoring and fault diagnosis technologies for wind turbines[J]. Automation of Electric Power Systems, 2023, 47(23): 55-69.
[24]	苏向敬, 周汶鑫, 李超杰, 等. 基于双重注意力LSTM神经网络的可解释海上风电出力预测[J]. 电力系统自动化, 2022, 46(7): 141-151.
	SU Xiangjing, ZHOU Wenxin, LI Chaojie, et al. Interpretable offshore wind power output forecasting based on long short-term memory neural network with dual-stage attention[J]. Automation of Electric Power Systems, 2022, 46(7): 141-151.
[25]	ZHANG K, ZHANG J, XU P D, et al. Explainable AI in deep reinforcement learning models for power system emergency control[J]. IEEE Transactions on Computational Social Systems, 2022, 9(2): 419-427.
[26]	ZHANG K, ZHANG J, XU P D, et al. A multi-hierarchical interpretable method for DRL-based dispatching control in power systems[J]. International Journal of Electrical Power & Energy Systems, 2023, 152: 109240.