关键词: 综合能源系统, 协调优化调度, 马尔可夫博弈, 能量路由器, 双层Actor-Critic网络, 优先经验回放机制。

Abstract: To ensure the economic operation of Integrated Energy Systems (IES), this paper addresses the challenges of traditional model-driven scheduling methods, such as difficulty in solving optimization models, slow convergence speed, and suboptimal results. A coordinated optimization scheduling method for IES based on energy routers is proposed.Firstly, the IES is divided into three regions using electrical, thermal, and cooling energy routers, and energy equipment is modeled to construct a Markov cooperative game model for IES optimization scheduling, forming a framework of cen-tralized training and distributed executionSecondly, a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm based on an improved bi-level Actor-Critic network is adopted. In this algorithm, a bi-level Critic network is used to evaluate action values to avoid overestimation, and a prioritized experience replay mechanism is introduced to improve the utilization of data in the experience replay buffer while maintaining data diversity.Finally, simulation re-sults verify that the proposed algorithm improves computation speed by 10.13 seconds and reduces daily scheduling costs by 1638.13 yuan compared to the unimproved version, achieving coordinated optimization scheduling of IES while ensuring system economic efficiency.

Key words: Integrated energy system, Coordination optimization scheduling, Markov game, Energy router, Dual-layer Actor-Critic network, Prioritized experience replay mechanism.