基于SC-SAC算法的REHMIS-IES优化调度策略

doi:10.3969/j.issn.2097-0706.2026.01.005

综合智慧能源 ›› 2026, Vol. 48 ›› Issue (1): 43-58.doi: 10.3969/j.issn.2097-0706.2026.01.005

• 综合智慧能源系统优化与调度 • 上一篇下一篇

基于SC-SAC算法的REHMIS-IES优化调度策略

潘雷(), 丁云飞(), 庞毅^*(), 王宇璇(), 陈建伟(), 高瑞(), 张立阳()

天津城建大学控制与机械工程学院，天津 300384

收稿日期:2025-08-12 修回日期:2025-10-21 出版日期:2026-01-25
通讯作者: *庞毅（1984），男，讲师，博士，从事综合能源系统规划方面的研究，primepang@163.com。
作者简介:潘雷（1981），男，教授，博士，从事新能源发电与电力电子技术、智能控制与智能系统等方面的研究，panlei4089@163.com；
丁云飞（2000），男，硕士生，从事综合能源系统、数学规划、深度强化学习等方面的研究，819667068@qq.com；
王宇璇（2001），女，硕士生，从事零能耗建筑、深度强化学习等方面的研究，1442030308@qq.com；
陈建伟（1981），女，讲师，博士，从事综合能源利用与可再生能源制氢等方面的研究，jianwei_chen1314@126.com；
高瑞（1984），男，讲师，博士，从事机器人导航与定位方面的研究，gaorui223@126.com；
张立阳（1986），男，副教授，博士，从事机器人控制与工程应用等方面的研究，zlysghr@163.com。
基金资助:
天津市重点研发计划项目(25YFXTHZ00530)

Optimal scheduling strategy for REHMIS-IES based on SC-SAC algorithm

PAN Lei(), DING Yunfei(), PANG Yi^*(), WANG Yuxuan(), CHEN Jianwei(), GAO Rui(), ZHANG Liyang()

School of Control and Mechanical Engineering， Tianjin Chengjian University， Tianjin 300384， China

Received:2025-08-12 Revised:2025-10-21 Published:2026-01-25
Supported by:
Key Research and Development Project of Tianjin(25YFXTHZ00530)

摘要/Abstract

摘要：

可再生能源-制氢-制甲醇一体站（REHMIS）通过利用可再生能源发电制取绿氢，并进一步将绿氢与二氧化碳合成甲醇，从而实现绿氢对传统化石能源制氢的替代。为了同时满足REHMIS的甲醇负荷需求及其配套建筑的多能源需求，设计了新型综合能源系统（IES）拓扑结构REHMIS-IES。为获得REHMIS-IES高效运行策略，提出了一种基于严格约束的软演员-评论家（SC-SAC）算法执行框架。将所建数学模型转化为马尔可夫决策过程，同时引入状态约束机制（SCM）以避免储能系统状态出现剧烈波动。在SC-SAC算法的执行阶段，将训练后的Q网络与动作约束转化成混合整数线性规划（MILP）模型，以保证调度决策能够满足各项运行约束。多场景仿真结果表明：所提系统在保障多能需求的同时可有效降低运行成本；与其他深度强化学习算法相比，SC-SAC算法可使系统能量不平衡度降低约16.2%，运行成本至少下降11.7%。

关键词: 可再生能源-制氢-制甲醇一体化站, 绿氢, 储能, 综合能源系统, 深度强化学习, 状态约束机制, 软演员-评论家算法, 混合整数线性规划

Abstract:

The renewable energy-hydrogen-methanol integrated station（REHMIS） produces green hydrogen using electricity generated from renewable energy sources， and further synthesizes methanol from the green hydrogen and carbon dioxide， thereby achieving the substitution of green hydrogen for hydrogen produced from conventional fossil fuels. To simultaneously meet the methanol load demand of REHMIS and the multi-energy demand of its supporting buildings， a novel integrated energy system（IES） topology named REHMIS-IES was designed. To obtain an efficient operation strategy for REHMIS-IES， an execution framework based on the strictly constrained soft actor-critic（SC-SAC） algorithm was proposed. The established mathematical model was transformed into a Markov decision process， and a state constraint mechanism（SCM） was incorporated to prevent drastic fluctuations in the state of the energy storage system. In the execution stage of the SC-SAC algorithm， the trained Q-network and action constraints were transformed into a mixed-integer linear programming（MILP） model to ensure that scheduling decisions could comply with all operational constraints. The results from multi-scenario simulations showed that the proposed system could effectively reduce operating costs while meeting multi-energy demands. Compared with other deep reinforcement learning algorithms， the SC-SAC algorithm could lower the system energy imbalance by approximately 16.2% and reduce operating costs by at least 11.7%.

Key words: renewable energy-hydrogen-methanol integrated station, green hydrogen, energy storage, integrated energy system, deep reinforcement learning, state constraint mechanism, soft actor-critic algorithm, mixed-integer linear programming

中图分类号:

TK01

潘雷, 丁云飞, 庞毅, 王宇璇, 陈建伟, 高瑞, 张立阳. 基于SC-SAC算法的REHMIS-IES优化调度策略[J]. 综合智慧能源, 2026, 48(1): 43-58.

PAN Lei, DING Yunfei, PANG Yi, WANG Yuxuan, CHEN Jianwei, GAO Rui, ZHANG Liyang. Optimal scheduling strategy for REHMIS-IES based on SC-SAC algorithm[J]. Integrated Intelligent Energy, 2026, 48(1): 43-58.

图/表 20

图1

图2

图3

图4

表1

系统设备参数

设备	参数	数值
WT	$P w t r$ /kW	800
	v_in/(m·s^-1)	3.5
	v_out/(m·s^-1)	25.0
	v_r/(m·s^-1)	12.0
	n_wt	2
PV	$P p v r$ /kW	0.265
	f_pv/%	80
	E_ref/(kW·m^-2)	1
	δ/(%·℃^-1)	0.47
	t_ref/℃	25
EL	r₁/(Ω·m²)	3.538 55×10^-4
	r₂/[Ω·m²)·℃^-1]	-3.021 50×10^-6
	s/V	2.239 6×10^-1
	t₁/(m²·A^-1)	5.130 93
	t₂/[(m²·℃)·A^-1]	-2.404 47×10²
	t₃/[(m²·℃)·A^-1]	5.995 76×10³
	A/m²	0.25
	f₁/(mA²·cm^-4)	250
	f₂	0.96
	F/(C·mol^-1)	96 485
HT	m_HTN/kg	800
HT	S_{HT_min}/S_{HT_max}/%	0/90
压缩机	c/[kJ·(kg·K)^-1]	14.304
	T₁/K	293
	η_c	0.75
	κ	1.4
HST	Q_HSTN/(kW·h)	1 000
HST	S_{HST_in}/S_{HST_out}/%	0/90
甲醇合成装置	$P M e O H u$ /(MW·h)	0.169
AC	$Q A C r$ /(kW·h)	750

表1

表2

表3

图5

图6

图7

图8

图9

表4

图10

图11

图12

图13

图14

图15

图16

参考文献 34

[1]	张普育. 双碳背景下风光储一体化可再生能源绿色标准化发展及实践[J]. 中国标准化, 2024(S2): 72-76.
	ZHANG Puyu. Development and practice of green standardization of renewable energy with integration of wind, solar energy and storage under the background of double carbon[J]. China Standardization, 2024(S2): 72-76.
[2]	武魏楠. 风光氢融合,开启双万亿级绿色燃料市场[J]. 能源, 2025(3): 62-64.
	WU Weinan. Wind, light and hydrogen fusion, opening up the double trillion-level green fuel market[J]. Energy, 2025(3): 62-64.
[3]	张晓峰, 艾芊, 陈旻昱. 基于风-氢-甲醇-碳捕集一体化的综合能源系统经济运行建模分析[J]. 现代电力, 2024, 41(4): 699-709.
	ZHANG Xiaofeng, AI Qian, CHEN Minyu. Economic operation modeling analysis of integrated energy system based on integration of wind-hydrogen-methanol-carbon capture[J]. Modern Electric Power, 2024, 41(4): 699-709.
[4]	王金龙. 考虑电转甲醇技术的园区级综合能源系统优化配置研究[J]. 节能, 2023, 42(11): 78-80.
	WANG Jinlong. Study on optimal allocation of park-level comprehensive energy system considering electricity-to-methanol technology[J]. Energy Conservation, 2023, 42(11): 78-80.
[5]	张润之, 周家辉, 梁士兴, 等. 离网式风光氢醇一体化系统容量配置运行调度优化及经济性分析[J]. 热力发电, 2024, 53(2): 48-58.
	ZHANG Runzhi, ZHOU Jiahui, LIANG Shixing, et al. Capacity configuration-operation scheduling optimization and economic analysis of the off grid wind and solar hydrogen alcohol integrated system[J]. Thermal Power Generation, 2024, 53(2): 48-58.
[6]	吕志鹏, 宋振浩, 李立生, 等. 含电动汽车的工业园区综合能源系统优化调度[J]. 中国电力, 2024, 57(4): 25-31.
	LYU Zhipeng, SONG Zhenhao, LI Lisheng, et al. Optimization scheduling of integrated energy system scheduling in industrial park containing electric vehicles[J]. Electric Power, 2024, 57(4): 25-31.
[7]	燕兵, 杨志林, 张岩, 等. 含海上风电制氢和多重响应的综合能源系统源-荷多时间尺度优化调度[J]. 南方电网技术, 2025, 19(6): 105-118. doi: 10.13648/j.cnki.issn1674-0629.2025.06.010
	YAN Bing, YANG Zhilin, ZHANG Yan, et al. Optimization of source-charge multi-time scale for hydrogen integrated energy system including hydrogen production from offshore wind power and multiple responses[J]. Southern Power System Technology, 2025, 19(6):105-118. doi: 10.13648/j.cnki.issn1674-0629.2025.06.010
[8]	安源, 李洋, 赵亭玉, 等. 考虑氨能多元利用与混合储能的综合能源系统双层优化调度[J]. 太阳能学报, 2025, 46(6): 89-98.
	AN Yuan, LI Yang, ZHAO Tingyu, et al. Double-layer optimal scheduling of integrated energy system considering ammonia energy multiple utilization and hybrid energy storage[J]. Acta Energiae Solaris Sinica, 2025, 46(6): 89-98.
[9]	谭甲群, 吕如轩, 鞠洪晋, 等. 基于改进秃鹰搜索算法的风光火储综合能源系统优化调度策略研究[J]. 综合智慧能源, 2025, 47(8): 68-76. doi: 10.3969/j.issn.2097-0706.2025.08.008
	TAN Jiaqun, RuxuanLYU, JU Hongjin, et al. Research on optimal scheduling strategy of wind-photovoltaic-thermal-storage integrated energy system based on IBES[J]. Integrated Intelligent Energy, 2025, 47(8): 68-76. doi: 10.3969/j.issn.2097-0706.2025.08.008
[10]	REN P, DONG Y C, ZHANG H L, et al. A unified robust planning framework for hydrogen energy multi-scale regulation of integrated energy system[J]. Energy, 2025, 314: 134325. doi: 10.1016/j.energy.2024.134325
[11]	李天明, 王小君, 窦嘉铭, 等. 基于约束强化学习的综合能源系统优化调度研究[J]. 电力系统保护与控制, 2025, 53(6): 1-14.
	LI Tianming, WANG Xiaojun, DOU Jiaming, et al. Research on optimal dispatch of integrated energy systems based on constrained reinforcement learning[J]. Power System Protection and Control, 2025, 53(6): 1-14.
[12]	LIU F, LIU Q Y, TAO Q, et al. Deep reinforcement learning based energy storage management strategy considering prediction intervals of wind power[J]. International Journal of Electrical Power & Energy Systems, 2023, 145: 108608. doi: 10.1016/j.ijepes.2022.108608
[13]	杨挺, 刘豪, 王静, 等. 基于深度强化学习的园区综合能源系统低碳经济调度[J]. 电网技术, 2024, 48(9): 3604-3613.
	YANG Ting, LIU Hao, WANG Jing, et al. Deep reinforcement learning-based low-carbon economic dispatch of park integrated energy system[J]. Power System Technology, 2024, 48(9): 3604-3613.
[14]	梁涛, 柴露露, 谭建鑫, 等. 基于深度强化学习算法的氢耦合电-热综合能源系统优化调度[J]. 电力自动化设备, 2025, 45(1): 59-66.
	LIANG Tao, CHAI Lulu, TAN Jianxin, et al. Optimal scheduling of hydrogen coupled electrothermal integrated energy system based on deep reinforcement learning algorithm[J]. Electric Power Automation Equipment, 2025, 45(1): 59-66.
[15]	包义辛, 徐椤赟, 杨强. 基于MADDPG算法的建筑群柔性负荷优化调控方法[J]. 综合智慧能源, 2023, 45(7): 61-69. doi: 10.3969/j.issn.2097-0706.2023.07.007
	BAO Yixin, XU Luoyun, YANG Qiang. Optimized control method for flexible load of a building complex based on MADDPG reinforcement learning[J]. Integrated Intelligent Energy, 2023, 45(7): 61-69. doi: 10.3969/j.issn.2097-0706.2023.07.007
[16]	DONG Y C, ZHANG H H, WANG C, et al. Soft actor-critic DRL algorithm for interval optimal dispatch of integrated energy systems with uncertainty in demand response and renewable energy[J]. Engineering Applications of Artificial Intelligence, 2024, 127: 107230. doi: 10.1016/j.engappai.2023.107230
[17]	李明扬, 窦梦园. 基于强化学习的含电动汽车虚拟电厂优化调度[J]. 综合智慧能源, 2024, 46(6): 27-34. doi: 10.3969/j.issn.2097-0706.2024.06.004
	LI Mingyang, DOU Mengyuan. Optimal scheduling of virtual power plants integrating electric vehicles based on reinforcement learning[J]. Integrated Intelligent Energy, 2024, 46(6): 27-34. doi: 10.3969/j.issn.2097-0706.2024.06.004
[18]	张磊, 吴红斌, 何叶, 等. 基于深度强化学习的氢能综合能源系统优化调度方法[J]. 电力系统自动化, 2024, 48(16): 132-141.
	ZHANG Lei, WU Hongbin, HE Ye, et al. Optimal scheduling method of hydrogen comprehensive energy system based on deep reinforcement learning[J]. Automation of Electric Power Systems, 2024, 48(16): 132-141.
[19]	LU Y, XIANG Y, HUANG Y, et al. Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load[J]. Energy, 2023, 271: 127087. doi: 10.1016/j.energy.2023.127087
[20]	LIANG T, ZHANG X C, TAN J X, et al. Deep reinforcement learning-based optimal scheduling of integrated energy systems for electricity, heat, and hydrogen storage[J]. Electric Power Systems Research, 2024, 233:110480. doi: 10.1016/j.epsr.2024.110480
[21]	王永利, 向皓, 李淑清. 考虑冷热电三联供系统耦合特性的区域综合能源系统运行优化[J]. 科学技术与工程, 2023, 23(18): 7787-7797.
	WANG Yongli, XIANG Hao, LI Shuqing. Regional integrated energy system considering combined cooling heating and power system coupling characteristics on operation optimization[J]. Science Technology and Engineering, 2023, 23(18): 7787-7797.
[22]	郑昊宇, 周家辉, 仝冰, 等. 配置压缩空气储能的绿氢系统容量配置和运行调度优化研究[J]. 综合智慧能源, 2025, 47(7): 64-70. doi: 10.3969/j.issn.2097-0706.2025.07.007
	ZHENG Haoyu, ZHOU Jiahui, TONG Bing, et al. Research on capacity allocation and operation scheduling optimization of green hydrogen system with compressed air energy storage[J]. Integrated Intelligent Energy, 2025, 47(7): 64-70. doi: 10.3969/j.issn.2097-0706.2025.07.007
[23]	ALI F, AHMAR M, JIANG Y X, et al. A techno-economic assessment of hybrid energy systems in rural Pakistan[J]. Energy, 2021, 215: 119103. DOI:10.1016/j.energy.2020.119103.
[24]	GHORBANI B, ZENDEHBOUDI S, MORADI M. Development of an integrated structure of hydrogen and oxygen liquefaction cycle using wind turbines, Kalina power generation cycle, and electrolyzer[J]. Energy, 2021, 221: 119653. doi: 10.1016/j.energy.2020.119653
[25]	LIANG T, CHEN M J, TAN J X, et al. Large-scale off-grid wind power hydrogen production multi-tank combination operation law and scheduling strategy taking into account alkaline electrolyzer characteristics[J]. Renewable Energy, 2024, 232: 121122. doi: 10.1016/j.renene.2024.121122
[26]	BHOGILLA S S, NIYAS H. Design of a hydrogen compressor for hydrogen fueling stations[J]. International Journal of Hydrogen Energy, 2019, 44(55): 29329-29337. doi: 10.1016/j.ijhydene.2019.02.171
[27]	岑增光, 耿斌, 高明海, 等. 考虑天然气混氢的园区综合能源系统电制氢优化配置[J]. 电力工程技术, 2024, 43(2): 55-64.
	CEN Zengguang, GENG Bin, GAO Minghai, et al. Optimal configuration of P2H in the park integrated energy system considering natural gas mixed with hydrogen[J]. Electric Power Engineering Technology, 2024, 43(2): 55-64.
[28]	ZHOU Y, WANG J J, LIU Y, et al. Incorporating deep learning of load predictions to enhance the optimal active energy management of combined cooling, heating and power system[J]. Energy, 2021, 233: 121134. doi: 10.1016/j.energy.2021.121134
[29]	CECCON F, JALVING J, HADDAD J, et al. OMLT: optimization & machine learning toolkit[J]. Journal of Machine Learning Research, 2022,23:1-8.
[30]	FISCHETTI M, JO J. Deep neural networks and mixed integer linear optimization[J]. Constraints, 2018, 23(3): 296-309. doi: 10.1007/s10601-018-9285-6
[31]	Optimization Gurobi. Gurobi官方网站[EB/OL]. [2025-07-31]. https://www.gurobi.com.
[32]	Department of Energy. Commercial reference buildings[EB/OL].[2025-07-31]. https://www.energy.gov/eere/buildings/commercial-reference-buildings.
[33]	PEEL M C, FINLAYSON B L, MCMAHON T A. Updated world map of the Köppen-Geiger climate classification[J]. Hydrology & Earth System Sciences, 2007, 11(3):259-263.
[34]	陈志鼎, 董亿, 滕明月. 考虑风-光不确定性的风-光-蓄-火联合调度研究[J]. 中国农村水利水电, 2024(8): 208-215. doi: 10.12396/znsd.232067
	CHEN Zhiding, DONG Yi, TENG Mingyue. Hybrid generation of wind-solar-pumped storage-thermal systems considering wind-solar uncertainties[J]. China Rural Water and Hydropower, 2024(8): 208-215. doi: 10.12396/znsd.232067

项目			价格
电网电价/ [元·(kW·h)^-1]	夏季(6—8月)	高峰	0.50
	夏季(6—8月)	低谷	0.31
	冬季(12—2月)	高峰	0.79
	冬季(12—2月)	低谷	0.31
	春秋季(3—5月，9 —11月)	高峰	0.69
	春秋季(3—5月，9 —11月)	低谷	0.31
平均购电价格/[元·(kW·h)^-1]			0.51
平均售电价格/[元·(kW·h)^-1]			0.29
CO₂采购价格/(元·kg^-1)			0.10

参数	数值
软更新系数	0.01
经验池容量	45 000
批量大小	256
训练次数	3 000
探索率	0.000 1
策略网络结构	3×64（ReLU,tanh）
评价网络结构	3×64（ReLU）

场景	运行成本/元	电网买电/元	电网卖电/元	热不平衡/(kW·h)	氢不平衡/kg
1	738.58	418.75	40.75	-256.18	-41.38
2	314.42	344.38	178.00	-278.85	0
3	596.16	450.50	113.62	0	-37.85
4	410.76	185.33	56.52	0	0
5	421.13	190.80	56.74	0	0
6	389.52	277.63	114.98	0	0

基于SC-SAC算法的REHMIS-IES优化调度策略

Optimal scheduling strategy for REHMIS-IES based on SC-SAC algorithm

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 20

参考文献 34

相关文章 15

编辑推荐

Metrics

本文评价

[1]	徐聪, 徐静静, 江婷, 薛东, 闫立辰. 数据驱动的综合能源系统运行优化研究[J]. 综合智慧能源, 2026, 48(1): 34-42.
[2]	薛东, 徐静静, 江婷, 王晓海, 徐聪. 基于双重特征处理的园区综合能源系统供热负荷预测研究[J]. 综合智慧能源, 2026, 48(1): 59-66.
[3]	梁富光, 马忠强. 基于评价因子重构与DECN-BiGRU的海岛微电网负荷预测[J]. 综合智慧能源, 2026, 48(1): 85-97.
[4]	窦翔, 李卓群, 张哲, 温鑫, 赵勃, 韩燕, 仲声. 基于CNN-BiLSTM-RF-KDE的综合能源系统负荷预测[J]. 综合智慧能源, 2025, 47(9): 60-70.
[5]	张元曦, 杨国华, 马龙腾, 马鑫, 刘耀泽. 基于改进DE算法的园区微电网风光储优化配置[J]. 综合智慧能源, 2025, 47(9): 71-79.
[6]	周凯, 吴炎喜, 黄宇翔, 杨景豪, 樊小朝, 李建伟, 魏志宗, 滕建, 陈历, 叶琴, 张浩, 姜军南. 大海拔高差地区固体流大规模重力储能技术[J]. 综合智慧能源, 2025, 47(8): 1-9.
[7]	甄文喜, 马喜平, 戴跃洪, 牛炜, 陈柏旭, 曾桂. 基于中间层控制的风-储自同步调频策略研究[J]. 综合智慧能源, 2025, 47(8): 21-29.
[8]	褚龙浩, 李笑竹. 共享储能参与下的多类型电能交易投标策略及价值分配机制[J]. 综合智慧能源, 2025, 47(8): 30-39.
[9]	谭甲群, 吕如轩, 鞠洪晋, 洪春雪, 肖海平, 雷兢, 韩振兴. 基于改进秃鹰搜索算法的风光火储综合能源系统优化调度策略研究[J]. 综合智慧能源, 2025, 47(8): 68-76.
[10]	宋坤, 顾文波. 考虑不确定性的综合能源系统建模及优化研究进展[J]. 综合智慧能源, 2025, 47(7): 32-43.
[11]	陈亮, 刘桂英, 粟时平, 唐长久, 王辰浩, 郭思桐. 基于双层网络PER-MADDPG算法的综合能源系统协调优化调度[J]. 综合智慧能源, 2025, 47(7): 44-54.
[12]	郑昊宇, 周家辉, 仝冰, 王海鸣, 徐钢, 张资悦. 配置压缩空气储能的绿氢系统容量配置和运行调度优化研究[J]. 综合智慧能源, 2025, 47(7): 64-70.
[13]	邢作霞, 赵子逸, 孙浩, 张鹏飞, 付启桐. 基于电制氢多级利用的综合能源系统低碳经济运行优化研究[J]. 综合智慧能源, 2025, 47(7): 71-81.
[14]	张德斌, 林文野, 何梓轩, 岳心如, 宋文吉, 冯自平, FARID Mohammed Mehdi, USHAK DE GRAGEDA Svetlana Nikolaevna. 计及广义储能的建筑产消者社区综合能源系统日前调度优化[J]. 综合智慧能源, 2025, 47(7): 82-92.
[15]	王成, 邵冲, 何欣, 董海鹰. 基于MOIBKA算法的电化学储能电站最优功率分配[J]. 综合智慧能源, 2025, 47(6): 74-84.