Optimal scheduling strategy for REHMIS-IES based on SC-SAC algorithm

doi:10.3969/j.issn.2097-0706.2026.01.005

Abstract

Abstract:

The renewable energy-hydrogen-methanol integrated station（REHMIS） produces green hydrogen using electricity generated from renewable energy sources， and further synthesizes methanol from the green hydrogen and carbon dioxide， thereby achieving the substitution of green hydrogen for hydrogen produced from conventional fossil fuels. To simultaneously meet the methanol load demand of REHMIS and the multi-energy demand of its supporting buildings， a novel integrated energy system（IES） topology named REHMIS-IES was designed. To obtain an efficient operation strategy for REHMIS-IES， an execution framework based on the strictly constrained soft actor-critic（SC-SAC） algorithm was proposed. The established mathematical model was transformed into a Markov decision process， and a state constraint mechanism（SCM） was incorporated to prevent drastic fluctuations in the state of the energy storage system. In the execution stage of the SC-SAC algorithm， the trained Q-network and action constraints were transformed into a mixed-integer linear programming（MILP） model to ensure that scheduling decisions could comply with all operational constraints. The results from multi-scenario simulations showed that the proposed system could effectively reduce operating costs while meeting multi-energy demands. Compared with other deep reinforcement learning algorithms， the SC-SAC algorithm could lower the system energy imbalance by approximately 16.2% and reduce operating costs by at least 11.7%.

Key words: renewable energy-hydrogen-methanol integrated station, green hydrogen, energy storage, integrated energy system, deep reinforcement learning, state constraint mechanism, soft actor-critic algorithm, mixed-integer linear programming

CLC Number:

TK01

PAN Lei, DING Yunfei, PANG Yi, WANG Yuxuan, CHEN Jianwei, GAO Rui, ZHANG Liyang. Optimal scheduling strategy for REHMIS-IES based on SC-SAC algorithm[J]. Integrated Intelligent Energy, 2026, 48(1): 43-58.

Figures/Tables 20

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

System equipment parameters

设备	参数	数值
WT	$P w t r$ /kW	800
	v_in/(m·s^-1)	3.5
	v_out/(m·s^-1)	25.0
	v_r/(m·s^-1)	12.0
	n_wt	2
PV	$P p v r$ /kW	0.265
	f_pv/%	80
	E_ref/(kW·m^-2)	1
	δ/(%·℃^-1)	0.47
	t_ref/℃	25
EL	r₁/(Ω·m²)	3.538 55×10^-4
	r₂/[Ω·m²)·℃^-1]	-3.021 50×10^-6
	s/V	2.239 6×10^-1
	t₁/(m²·A^-1)	5.130 93
	t₂/[(m²·℃)·A^-1]	-2.404 47×10²
	t₃/[(m²·℃)·A^-1]	5.995 76×10³
	A/m²	0.25
	f₁/(mA²·cm^-4)	250
	f₂	0.96
	F/(C·mol^-1)	96 485
HT	m_HTN/kg	800
HT	S_{HT_min}/S_{HT_max}/%	0/90
压缩机	c/[kJ·(kg·K)^-1]	14.304
	T₁/K	293
	η_c	0.75
	κ	1.4
HST	Q_HSTN/(kW·h)	1 000
HST	S_{HST_in}/S_{HST_out}/%	0/90
甲醇合成装置	$P M e O H u$ /(MW·h)	0.169
AC	$Q A C r$ /(kW·h)	750

Table 1

Table 2

Table 3

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Table 4

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

References 34

[1]	张普育. 双碳背景下风光储一体化可再生能源绿色标准化发展及实践[J]. 中国标准化, 2024(S2): 72-76.
	ZHANG Puyu. Development and practice of green standardization of renewable energy with integration of wind, solar energy and storage under the background of double carbon[J]. China Standardization, 2024(S2): 72-76.
[2]	武魏楠. 风光氢融合,开启双万亿级绿色燃料市场[J]. 能源, 2025(3): 62-64.
	WU Weinan. Wind, light and hydrogen fusion, opening up the double trillion-level green fuel market[J]. Energy, 2025(3): 62-64.
[3]	张晓峰, 艾芊, 陈旻昱. 基于风-氢-甲醇-碳捕集一体化的综合能源系统经济运行建模分析[J]. 现代电力, 2024, 41(4): 699-709.
	ZHANG Xiaofeng, AI Qian, CHEN Minyu. Economic operation modeling analysis of integrated energy system based on integration of wind-hydrogen-methanol-carbon capture[J]. Modern Electric Power, 2024, 41(4): 699-709.
[4]	王金龙. 考虑电转甲醇技术的园区级综合能源系统优化配置研究[J]. 节能, 2023, 42(11): 78-80.
	WANG Jinlong. Study on optimal allocation of park-level comprehensive energy system considering electricity-to-methanol technology[J]. Energy Conservation, 2023, 42(11): 78-80.
[5]	张润之, 周家辉, 梁士兴, 等. 离网式风光氢醇一体化系统容量配置运行调度优化及经济性分析[J]. 热力发电, 2024, 53(2): 48-58.
	ZHANG Runzhi, ZHOU Jiahui, LIANG Shixing, et al. Capacity configuration-operation scheduling optimization and economic analysis of the off grid wind and solar hydrogen alcohol integrated system[J]. Thermal Power Generation, 2024, 53(2): 48-58.
[6]	吕志鹏, 宋振浩, 李立生, 等. 含电动汽车的工业园区综合能源系统优化调度[J]. 中国电力, 2024, 57(4): 25-31.
	LYU Zhipeng, SONG Zhenhao, LI Lisheng, et al. Optimization scheduling of integrated energy system scheduling in industrial park containing electric vehicles[J]. Electric Power, 2024, 57(4): 25-31.
[7]	燕兵, 杨志林, 张岩, 等. 含海上风电制氢和多重响应的综合能源系统源-荷多时间尺度优化调度[J]. 南方电网技术, 2025, 19(6): 105-118. doi: 10.13648/j.cnki.issn1674-0629.2025.06.010
	YAN Bing, YANG Zhilin, ZHANG Yan, et al. Optimization of source-charge multi-time scale for hydrogen integrated energy system including hydrogen production from offshore wind power and multiple responses[J]. Southern Power System Technology, 2025, 19(6):105-118. doi: 10.13648/j.cnki.issn1674-0629.2025.06.010
[8]	安源, 李洋, 赵亭玉, 等. 考虑氨能多元利用与混合储能的综合能源系统双层优化调度[J]. 太阳能学报, 2025, 46(6): 89-98.
	AN Yuan, LI Yang, ZHAO Tingyu, et al. Double-layer optimal scheduling of integrated energy system considering ammonia energy multiple utilization and hybrid energy storage[J]. Acta Energiae Solaris Sinica, 2025, 46(6): 89-98.
[9]	谭甲群, 吕如轩, 鞠洪晋, 等. 基于改进秃鹰搜索算法的风光火储综合能源系统优化调度策略研究[J]. 综合智慧能源, 2025, 47(8): 68-76. doi: 10.3969/j.issn.2097-0706.2025.08.008
	TAN Jiaqun, RuxuanLYU, JU Hongjin, et al. Research on optimal scheduling strategy of wind-photovoltaic-thermal-storage integrated energy system based on IBES[J]. Integrated Intelligent Energy, 2025, 47(8): 68-76. doi: 10.3969/j.issn.2097-0706.2025.08.008
[10]	REN P, DONG Y C, ZHANG H L, et al. A unified robust planning framework for hydrogen energy multi-scale regulation of integrated energy system[J]. Energy, 2025, 314: 134325. doi: 10.1016/j.energy.2024.134325
[11]	李天明, 王小君, 窦嘉铭, 等. 基于约束强化学习的综合能源系统优化调度研究[J]. 电力系统保护与控制, 2025, 53(6): 1-14.
	LI Tianming, WANG Xiaojun, DOU Jiaming, et al. Research on optimal dispatch of integrated energy systems based on constrained reinforcement learning[J]. Power System Protection and Control, 2025, 53(6): 1-14.
[12]	LIU F, LIU Q Y, TAO Q, et al. Deep reinforcement learning based energy storage management strategy considering prediction intervals of wind power[J]. International Journal of Electrical Power & Energy Systems, 2023, 145: 108608. doi: 10.1016/j.ijepes.2022.108608
[13]	杨挺, 刘豪, 王静, 等. 基于深度强化学习的园区综合能源系统低碳经济调度[J]. 电网技术, 2024, 48(9): 3604-3613.
	YANG Ting, LIU Hao, WANG Jing, et al. Deep reinforcement learning-based low-carbon economic dispatch of park integrated energy system[J]. Power System Technology, 2024, 48(9): 3604-3613.
[14]	梁涛, 柴露露, 谭建鑫, 等. 基于深度强化学习算法的氢耦合电-热综合能源系统优化调度[J]. 电力自动化设备, 2025, 45(1): 59-66.
	LIANG Tao, CHAI Lulu, TAN Jianxin, et al. Optimal scheduling of hydrogen coupled electrothermal integrated energy system based on deep reinforcement learning algorithm[J]. Electric Power Automation Equipment, 2025, 45(1): 59-66.
[15]	包义辛, 徐椤赟, 杨强. 基于MADDPG算法的建筑群柔性负荷优化调控方法[J]. 综合智慧能源, 2023, 45(7): 61-69. doi: 10.3969/j.issn.2097-0706.2023.07.007
	BAO Yixin, XU Luoyun, YANG Qiang. Optimized control method for flexible load of a building complex based on MADDPG reinforcement learning[J]. Integrated Intelligent Energy, 2023, 45(7): 61-69. doi: 10.3969/j.issn.2097-0706.2023.07.007
[16]	DONG Y C, ZHANG H H, WANG C, et al. Soft actor-critic DRL algorithm for interval optimal dispatch of integrated energy systems with uncertainty in demand response and renewable energy[J]. Engineering Applications of Artificial Intelligence, 2024, 127: 107230. doi: 10.1016/j.engappai.2023.107230
[17]	李明扬, 窦梦园. 基于强化学习的含电动汽车虚拟电厂优化调度[J]. 综合智慧能源, 2024, 46(6): 27-34. doi: 10.3969/j.issn.2097-0706.2024.06.004
	LI Mingyang, DOU Mengyuan. Optimal scheduling of virtual power plants integrating electric vehicles based on reinforcement learning[J]. Integrated Intelligent Energy, 2024, 46(6): 27-34. doi: 10.3969/j.issn.2097-0706.2024.06.004
[18]	张磊, 吴红斌, 何叶, 等. 基于深度强化学习的氢能综合能源系统优化调度方法[J]. 电力系统自动化, 2024, 48(16): 132-141.
	ZHANG Lei, WU Hongbin, HE Ye, et al. Optimal scheduling method of hydrogen comprehensive energy system based on deep reinforcement learning[J]. Automation of Electric Power Systems, 2024, 48(16): 132-141.
[19]	LU Y, XIANG Y, HUANG Y, et al. Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load[J]. Energy, 2023, 271: 127087. doi: 10.1016/j.energy.2023.127087
[20]	LIANG T, ZHANG X C, TAN J X, et al. Deep reinforcement learning-based optimal scheduling of integrated energy systems for electricity, heat, and hydrogen storage[J]. Electric Power Systems Research, 2024, 233:110480. doi: 10.1016/j.epsr.2024.110480
[21]	王永利, 向皓, 李淑清. 考虑冷热电三联供系统耦合特性的区域综合能源系统运行优化[J]. 科学技术与工程, 2023, 23(18): 7787-7797.
	WANG Yongli, XIANG Hao, LI Shuqing. Regional integrated energy system considering combined cooling heating and power system coupling characteristics on operation optimization[J]. Science Technology and Engineering, 2023, 23(18): 7787-7797.
[22]	郑昊宇, 周家辉, 仝冰, 等. 配置压缩空气储能的绿氢系统容量配置和运行调度优化研究[J]. 综合智慧能源, 2025, 47(7): 64-70. doi: 10.3969/j.issn.2097-0706.2025.07.007
	ZHENG Haoyu, ZHOU Jiahui, TONG Bing, et al. Research on capacity allocation and operation scheduling optimization of green hydrogen system with compressed air energy storage[J]. Integrated Intelligent Energy, 2025, 47(7): 64-70. doi: 10.3969/j.issn.2097-0706.2025.07.007
[23]	ALI F, AHMAR M, JIANG Y X, et al. A techno-economic assessment of hybrid energy systems in rural Pakistan[J]. Energy, 2021, 215: 119103. DOI:10.1016/j.energy.2020.119103.
[24]	GHORBANI B, ZENDEHBOUDI S, MORADI M. Development of an integrated structure of hydrogen and oxygen liquefaction cycle using wind turbines, Kalina power generation cycle, and electrolyzer[J]. Energy, 2021, 221: 119653. doi: 10.1016/j.energy.2020.119653
[25]	LIANG T, CHEN M J, TAN J X, et al. Large-scale off-grid wind power hydrogen production multi-tank combination operation law and scheduling strategy taking into account alkaline electrolyzer characteristics[J]. Renewable Energy, 2024, 232: 121122. doi: 10.1016/j.renene.2024.121122
[26]	BHOGILLA S S, NIYAS H. Design of a hydrogen compressor for hydrogen fueling stations[J]. International Journal of Hydrogen Energy, 2019, 44(55): 29329-29337. doi: 10.1016/j.ijhydene.2019.02.171
[27]	岑增光, 耿斌, 高明海, 等. 考虑天然气混氢的园区综合能源系统电制氢优化配置[J]. 电力工程技术, 2024, 43(2): 55-64.
	CEN Zengguang, GENG Bin, GAO Minghai, et al. Optimal configuration of P2H in the park integrated energy system considering natural gas mixed with hydrogen[J]. Electric Power Engineering Technology, 2024, 43(2): 55-64.
[28]	ZHOU Y, WANG J J, LIU Y, et al. Incorporating deep learning of load predictions to enhance the optimal active energy management of combined cooling, heating and power system[J]. Energy, 2021, 233: 121134. doi: 10.1016/j.energy.2021.121134
[29]	CECCON F, JALVING J, HADDAD J, et al. OMLT: optimization & machine learning toolkit[J]. Journal of Machine Learning Research, 2022,23:1-8.
[30]	FISCHETTI M, JO J. Deep neural networks and mixed integer linear optimization[J]. Constraints, 2018, 23(3): 296-309. doi: 10.1007/s10601-018-9285-6
[31]	Optimization Gurobi. Gurobi官方网站[EB/OL]. [2025-07-31]. https://www.gurobi.com.
[32]	Department of Energy. Commercial reference buildings[EB/OL].[2025-07-31]. https://www.energy.gov/eere/buildings/commercial-reference-buildings.
[33]	PEEL M C, FINLAYSON B L, MCMAHON T A. Updated world map of the Köppen-Geiger climate classification[J]. Hydrology & Earth System Sciences, 2007, 11(3):259-263.
[34]	陈志鼎, 董亿, 滕明月. 考虑风-光不确定性的风-光-蓄-火联合调度研究[J]. 中国农村水利水电, 2024(8): 208-215. doi: 10.12396/znsd.232067
	CHEN Zhiding, DONG Yi, TENG Mingyue. Hybrid generation of wind-solar-pumped storage-thermal systems considering wind-solar uncertainties[J]. China Rural Water and Hydropower, 2024(8): 208-215. doi: 10.12396/znsd.232067

项目			价格
电网电价/ [元·(kW·h)^-1]	夏季(6—8月)	高峰	0.50
	夏季(6—8月)	低谷	0.31
	冬季(12—2月)	高峰	0.79
	冬季(12—2月)	低谷	0.31
	春秋季(3—5月，9 —11月)	高峰	0.69
	春秋季(3—5月，9 —11月)	低谷	0.31
平均购电价格/[元·(kW·h)^-1]			0.51
平均售电价格/[元·(kW·h)^-1]			0.29
CO₂采购价格/(元·kg^-1)			0.10

参数	数值
软更新系数	0.01
经验池容量	45 000
批量大小	256
训练次数	3 000
探索率	0.000 1
策略网络结构	3×64（ReLU,tanh）
评价网络结构	3×64（ReLU）

场景	运行成本/元	电网买电/元	电网卖电/元	热不平衡/(kW·h)	氢不平衡/kg
1	738.58	418.75	40.75	-256.18	-41.38
2	314.42	344.38	178.00	-278.85	0
3	596.16	450.50	113.62	0	-37.85
4	410.76	185.33	56.52	0	0
5	421.13	190.80	56.74	0	0
6	389.52	277.63	114.98	0	0