Optimized scheduling on large-scale hydrogen production system for off-grid renewable energy based on DDPG algorithm

doi:10.3969/j.issn.2097-0706.2024.06.005

Abstract

Abstract:

To improve the renewable energy consumption， reduce the investment on rectifiers and grid connection equipment， cut down the cost of water electrolysis for hydrogen production through powering hydrogen production by renewable energy， an islanded renewable energy large-scale hydrogen production system is constructed. An intelligent energy management platform can improve the economy and safety of the system. Firstly， a simulation model of the renewable energy large-scale hydrogen production system is established and its control strategy is formulated. Secondly， an energy optimization scheduling strategy based on deep deterministic policy gradient （DDPG） algorithm is proposed. Through long-term trainings， the agent obtained from the DDPG algorithm can achieve intelligent dynamic optimized scheduling on energy. Comparing the performances of the proposed strategy with deep Q network （DQN）， Particle Swarm Optimization （PSO） and traditional control methods in terms of economy and safety， it is shown that applying the DDPG algorithm in energy optimization and management can get higher economic returns and utilization rates of renewable resources， and ensure the safe operation of the system.

Key words: renewable energy, large-scale hydrogen production, off-grid, deep deterministic policy gradient, optimized scheduling

CLC Number:

TK91：TM715

ZHENG Qingming, JING Yanwei, LIANG Tao, CHAI Lulu, LYU Liangnian. Optimized scheduling on large-scale hydrogen production system for off-grid renewable energy based on DDPG algorithm[J]. Integrated Intelligent Energy, 2024, 46(6): 35-43.

Figures/Tables 19

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Table 1

Table 2

Table 3

Table 4

References 17

[1]	ZHANG Q, CHEN W M, LING W. Policy optimization of hydrogen energy industry considering government policy preference in China[J]. Sustainable Production and Consumption, 2022, 33: 890-902.
[2]	韩世旺, 赵颖, 张兴宇, 等. 面向碳中和的新型电力系统氢储能调峰技术研究[J]. 综合智慧能源, 2022, 44(9):20-26. doi: 10.3969/j.issn.2097-0706.2022.09.003
	HAN Shiwang, ZHAO Ying, ZHANG Xingyu, et al. Researches on hydrogen storage peak‑shaving technology for new power systems to achieve carbon neutrality[J]. Integrated Intelligent Energy, 2022, 44(9):20-26. doi: 10.3969/j.issn.2097-0706.2022.09.003
[3]	HAO W, ZHANG S X, LI X X, et al. A multivariate coupled economic model study on hydrogen production by renewable energy combined with off‑peak electricity[J]. International Journal of Hydrogen Energy, 2022, 47(58):24481-24492.
[4]	葛磊蛟, 崔庆雪, 李明玮, 等. 风光波动性电源电解水制氢技术综述[J]. 综合智慧能源, 2022, 44(5):1-14. doi: 10.3969/j.issn.2097-0706.2022.05.001
	GE Leijiao, CUI Qingxue, LI Mingwei, et al. Review on water electrolysis for hydrogen production powered by fluctuating wind power and PV[J]. Integrated Intelligent Energy, 2022, 44(5):1-14. doi: 10.3969/j.issn.2097-0706.2022.05.001
[5]	KAFETZIS A, ZIOGOU C, Panopoulos K D, et al. Energy management strategies based on hybrid automata for islanded microgrids with renewable sources, batteries and hydrogen[J]. Renewable and Sustainable Energy Reviews, 2020, 134:110118.
[6]	MOHAMMED A, GHAITHAN A M, AL-HANBALI A, et al. A multi‑objective optimization model based on mixed integer linear programming for sizing a hybrid PV‑hydrogen storage system[J]. International Journal of Hydrogen Energy, 2023, 48(26):9748-9761.
[7]	FANG R M. Multi‑objective optimized operation of integrated energy system with hydrogen storage[J]. International Journal of Hydrogen Energy, 2019, 44(56):29409-29417.
[8]	CHENG T, ZHU X Q, GU X Y, et al. Stochastic energy management and scheduling of microgrids in correlated environment: A deep learning‑oriented approach[J]. Sustainable Cities and Society, 2021, 69:102856.
[9]	HARROLD D, CAO J, FAN Z. Renewable energy integration and microgrid energy trading using multi‑agent deep rein‑ forcement learning[J]. Applied Energy, 2022, 318:119151.
[10]	刘俊峰, 陈剑龙, 王晓生, 等. 基于深度强化学习的微能源网能量管理与优化策略研究[J]. 电网技术, 2020, 44(10):3794-3803.
	LIU Junfeng, CHEN Jianlong, WANG Xiaosheng, et al. Energy management and optimization of multi‑energy grid based on deep reinforcement learning[J]. Power System Technology, 2020, 44(10):3794-3803.
[11]	张自东, 邱才明, 张东霞, 等. 基于深度强化学习的微电网复合储能协调控制方法[J]. 电网技术, 2019, 43(6):1914-1921.
	ZHANG Zidong, QIU Caiming, ZHANG Dongxia, et al. A coordinated control method for hybrid energy storage system in microgrid based on deep reinforcement learning[J]. Power System Technology, 2019, 43(6):1914-1921.
[12]	余宏晖. 基于深度强化学习的微电网能量优化管理研究[D]. 广州: 华南理工大学, 2021.
	YU Honghui. Research on energy optimization management of microgrid based on deep reinforcement learning[D]. Guangzhou: South China University of Technology, 2021.
[13]	孙惠娟, 段伟男, 陈俐, 等. 基于乐观行动-评判深度强化学习的含氢综合能源系统低碳经济调度[J]. 电网技术, 2024, 48(5):1873-1883.
	SUN Huijuan, DUAN Weinan, CHEN Li, et al. Low‑carbon economic scheduling of hydrogen integrated energy system based on optimistic actor-critic deep reinforcement learning[J]. Power System Technology, 2024, 48(5):1873-1883.
[14]	梁涛, 孙博峰, 谭建鑫, 等. 基于深度强化学习算法的风光互补可再生能源制氢系统调度方案[J]. 高电压技术, 2023, 49(6):2264-2275.
	LIANG Tao, SUN Bofeng, Tan Jianxin, et al. Scheduling scheme of wind-solar complementary renewable energy hydrogen production system based on deep reinforcement learning[J]. High Voltage Engineering, 2023, 49(6):2264-2275.
[15]	彭刘阳, 孙元章, 徐箭, 等. 基于深度强化学习的自适应不确定性经济调度[J]. 电力系统自动化, 2020, 44(9):33-42.
	PENG Liuyang, SUN Yuanzhang, XU Jian, et al. Self‑adaptive uncertainty economic dispatch based on deep reinforcement learning[J]. Automation of Electric Power Systems, 2020, 44(9):33-42.
[16]	王桂兰, 张海晓, 刘宏, 等. 基于近端策略优化算法含碳捕集的综合能源系统低碳经济调度[J]. 计算机应用研究, 2024, 41(5):1508-1514.
	WANG Guilan, ZHANG Haixiao, LIU Hong, et al. Low carbon economic scheduling of integrated energy systems based on proximal policy optimization algorithm with carbon capture[J]. Application Research of Computers, 2024, 41(5):1508-1514.
[17]	江昌旭, 刘晨曦, 林铮, 等. 基于深度强化学习的电力系统暂态稳定控制策略研究综述[J]. 高电压技术, 2023, 49(12):5171-5186.
	JIANG Changxu, LIU Chenxi, LIN Zheng, et al. Review of power system transient stability control strategies based on deep reinforcement learning[J]. High Voltage Engineering, 2023, 49(12):5171-5186.

参数	控制策略	PSO	DQN	DDPG
运行成本/美元	314.30	287.77	252.35	236.65
损失费用/美元	599.64	559.64	723.03	516.28
卖氢收益/美元	4 000	4 000	4 000	4 000
总利润/美元	3 086.06	3 152.59	3 024.62	3 247.07
丢弃功率/kW	2 998.18	2 798.18	3 615.17	2 581.38
深度过充时间/h	9	3	0	0
深度过放时间/h	0	3	0	0

参数	控制策略	PSO	DQN	DDPG
运行成本/美元	216.72	203.64	181.15	181.55
损失费用/美元	0	26.37	231.16	0
卖氢收益/美元	3 000	3 000	3 000	3 000
总利润/美元	2 783.28	2 769.99	2 587.69	2 918.45
丢弃功率/kW	0	131.84	1 155.78	0
深度过充时间/h	0	0	0	0
深度过放时间/h	0	0	0	0

参数	控制策略	PSO	DQN	DDPG
运行成本/美元	204.38	204.38	175.2	182.92
损失费用/美元	234.44	234.44	318.17	62.00
卖氢收益/美元	3 000	3 000	3 000	3 000
总利润/美元	2 561.18	2 561.18	2 506.63	2 755.08
丢弃功率/kW	1 172.22	1 172.22	1 590.83	310.00
深度过充时间/h	16	16	0	0
深度过放时间/h	0	0	0	0

参数	控制策略	PSO	DQN	DDPG
运行成本/美元	296.31	296.31	261.64	297.01
损失费用/美元	400.01	400.01	416.51	136.02
卖氢收益/美元	4 000	4 000	4 000	4 000
总利润/美元	3 303.68	3 303.68	3 321.85	3 566.97
丢弃功率/kW	2 000.07	2 000.07	2 082.57	680.09
深度过充时间/h	6	6	0	0
深度过放时间/h	1	1	0	0