基于深度强化学习的风电场功率多变量综合优化控制

doi:10.3969/j.issn.2097-0706.2025.01.003

综合智慧能源 ›› 2025, Vol. 47 ›› Issue (1): 18-25.doi: 10.3969/j.issn.2097-0706.2025.01.003

• 基于AI的新型电力系统调度 • 上一篇下一篇

基于深度强化学习的风电场功率多变量综合优化控制

张华钦¹^,²(), 刘伟³^,^*(), 王慧¹^,², 李雷孝¹^,², 莎仁高娃⁴

1.内蒙古工业大学数据科学与应用学院，呼和浩特 010080
2.内蒙古自治区基于大数据的软件服务工程技术研究中心，呼和浩特 010080
3.内蒙古巴彦淖尔市临河区气象局，内蒙古巴彦淖尔 015000
4.内蒙古自治区公安厅，呼和浩特 010080

收稿日期:2024-10-08 修回日期:2024-11-13 出版日期:2025-01-25
通讯作者: *刘伟（1966），男，高级工程师，从事农业气象和气候资源开发利用方面的研究，723674266@qq.com。
作者简介:张华钦（2000），男，硕士生，从事可再生能源控制，深度强化学习方面的研究，20221800730@imut.edu.cn。
基金资助:
国家自然科学基金项目(62362055);内蒙古自治区气象局科技创新项目(nmqxkjcx202312);内蒙古自治区气象局科技创新项目(nmqxkjcx202435)

Multivariable integrated power control optimization of wind farms based on deep reinforcement learning

ZHANG Huaqin¹^,²(), LIU Wei³^,^*(), WANG Hui¹^,², LI Leixiao¹^,², Sharengaowa⁴

1. School of Data Science and Applications，Inner Mongolia University of Technology，Hohhot 010080，China
2. Inner Mongolia Autonomous Region Engineering & Technology Research Center of Big Data Based Software Service，Hohhot 010080， China
3. Linhe District Meteorological Bureau， Bayannur 015000，China
4. Public Security Department of Inner Mongolia Autonomous Region，Hohhot 010080，China

Received:2024-10-08 Revised:2024-11-13 Published:2025-01-25
Supported by:
National Natural Science Foundation of China(62362055);Scientific and Technological Innovation Project of Meteorological Bureau of Inner Mongolia Autonomous Region(nmqxkjcx202312);Scientific and Technological Innovation Project of Meteorological Bureau of Inner Mongolia Autonomous Region(nmqxkjcx202435)

摘要/Abstract

摘要：

我国提出“双碳”目标，提出构建以新能源为主体的新型电力系统。依据真实风电场的数据，通过研究优化控制策略，提升风电场输出功率，从而进一步提高风电的利用率。聚焦风电机组间尾流效应问题，提出一种基于无模型深度强化学习（DRL）的风电场功率多变量优化控制策略。该策略采用近端策略优化（PPO）算法对动态风电场中的风机偏航角、倾斜角、叶片桨距角和叶尖速比（TSR）多个变量进行优化，通过智能体在运行过程中产生的数据进行智能学习，从而得到最优控制策略，克服传统数学优化方法的局限性。仿真结果表明，与现有的风电机组控制算法相比，基于无模型DRL的多变量优化控制策略显著提高了计算效率，降低了参数优化难度，优化了尾流方向和强度，优化后平均输出功率提升37.08%。

关键词: 风电场, 深度强化学习, 尾流控制, 多变量控制, 功率优化, 近端策略优化, “双碳”目标, 新型电力系统

Abstract:

China has proposed the"dual carbon"strategy，aiming to build a new power system with renewable energy as the primary component.Based on real wind farm data，optimization control strategies were proposed to improve the wind farm's output power，thereby further enhancing wind energy utilization.The wake effects between wind turbines were the main focus，and a wind farm power multivariable optimization control strategy based on model-free deep reinforcement learning（DRL）was proposed.The strategy employed the Proximal Policy Optimization（PPO）algorithm to optimize multiple variables，including the yaw angle，tilt angle，blade pitch angle，and tip speed ratio（TSR） in a dynamic wind farm.Through intelligent agents learning from the data generated by the agent during operation，an optimal control strategy was obtained，overcoming the limitations of traditional mathematical optimization methods. Simulation results showed that，compared to existing wind turbine control algorithms，the model-free DRL-based multivariable optimization control strategy significantly improved computational efficiency，reduced the difficulty of parameter optimization，and optimized the direction and strength of the wake.The optimized average output power was increased by 37.08%.

Key words: wind farm, deep reinforcement learning, wake control, multivariable control, power optimization, PPO, "dual carbon" strategy, new power system

中图分类号:

TK81

张华钦, 刘伟, 王慧, 李雷孝, 莎仁高娃. 基于深度强化学习的风电场功率多变量综合优化控制[J]. 综合智慧能源, 2025, 47(1): 18-25.

ZHANG Huaqin, LIU Wei, WANG Hui, LI Leixiao, Sharengaowa. Multivariable integrated power control optimization of wind farms based on deep reinforcement learning[J]. Integrated Intelligent Energy, 2025, 47(1): 18-25.

图/表 12

图1

图2

图3

表1

风电场参数

参数	数值
风电机组直径 $/ m$	126
风电机组之间的距离 $/$ m	630
轮毂高度 $/ m$	90
空气密度 $/ (k g · m - 3)$	1.225
切入风速 $/ (m · s - 1)$	3
切出风速 $/ (m · s - 1)$	25

表1

图4

表2

图5

图6

图7

图8

图9

图10

参考文献 24

[1]	朱博, 吴水军, 黄柯昊, 等. 水风光互补系统一次调频方案研究[J]. 太阳能学报, 2024, 45(5):412-421.
	ZHU Bo, WU Shuijun, HUANG Kehao, et al. Research on primary frequency regulation scheme of water-wind-solar complementary system[J]. Acta Energiae Solaris Sinica, 2024, 45(5):412-421.
[2]	瞿国华. 我国能源转型与过渡能源的合理选择[J]. 科学发展, 2022(12):88-96.
	QU Guohua. China's rational choice of energy transition and transition energy[J]. Scientific Development, 2022(12):88-96.
[3]	KUMAR D, CHATTERJEE K. A review of conventional and advanced MPPT algorithms for wind energy systems[J]. Renewable and Sustainable Energy Reviews, 2016, 55:957-970.
[4]	宋冬然, 沈旭涛, 黄朝能, 等. 基于代理模型辅助改进标准粒子群算法的浮式风电场功率优化[J]. 中国电机工程学报, 2023, 43(S1):217-228.
	SONG Dongran, SHEN Xutao, HUANG Chaoneng, et al. Power optimization of floating wind farm based on improved standard particle swarm optimization assisted by proxy model[J]. Proceedings of the CSEE, 2023, 43(S1):217-228.
[5]	NASH R, NOURI R, VASEL A. Wind turbine wake control strategies:A review and concept proposal[J]. Energy Conversion and Management, 2021, 245:114581.
[6]	CAO L Y, WEI W, ZHANG J W, et al. Adaptive economic predictive control for offshore wind farm active yaw considering generation uncertainty[J]. Applied Energy, 2023, 351:121849.
[7]	李雄威, 徐家豪, 朱润泽, 等. 基于偏航尾流模型的风电场功率协同优化研究[J]. 太阳能学报, 2022, 43(10):144-151. doi: 10.19912/j.0254-0096.tynxb.2021-1292
	LI Xiongwei, XU Jiahao, ZHU Runze, et al. Study on power collaborative optimization of wind farm based on yaw wake model[J]. Acta Energiae Solaris Sinica, 2022, 43(10):144-151. doi: 10.19912/j.0254-0096.tynxb.2021-1292
[8]	SUN X X, LI Y M, ZHANG F, et al. Decentralized yaw optimization for maximizing wind farm production based on deep reinforcement learning[J]. Energy Conversion and Management, 2023, 286:1-11.
[9]	LIU Y, ZHANG J H, ZHOU J H. The influence of tilt angle on the aerodynamic performance of a wind turbine[J]. Applied Sciences, 2020, 10(15):5380.
[10]	XU X H, LIU D F, LI Y, et al. Collective large-scale wind farm multivariate power output control based on hierarchical communication multi-agent proximal policy optimization[J]. Renewable Energy, 2023, 219:119479.
[11]	SUTTON R S, BARTO A G. Reinforcement learning:An introduction[M]. MIT Press, 2018.
[12]	LI Y. Deep reinforcement learning:An overview[Z]. 2017.
[13]	王康平, 张兴科, 刘财华, 等. 基于自适应下垂控制的风电场无功电压控制策略[J]. 综合智慧能源, 2022, 44(4):12-19. doi: 10.3969/j.issn.2097-0706.2022.04.002
	WANG Kangping, ZHANG Xingke, LIU Caihua, et al. Reactive power and voltage control strategy based on adaptive droop control for wind power plants[J]. Integrated Intelligent Energy, 2022, 44(4):12-19. doi: 10.3969/j.issn.2097-0706.2022.04.002
[14]	刘玉山, 胡阔海, 王灵梅, 等. 考虑尾流效应的风电场输出功率优化[J]. 可再生能源, 2023, 41(10):1336-1342.
	LIU Yushan, HU Kuohai, WANG Lingmei, et al. Optimization of wind farm output power considering wake effect[J]. Renewable Energy Resources, 2023, 41(10):1336-1342.
[15]	PADULLAPARTHI V R, NAGARATHINAM S, VASAN A, et al. Falcon-farm Level control for wind turbines using multi-agent deep reinforcement learning[J]. Renewable Energy, 2022, 181:445-456.
[16]	HAI B V, THANH N T, MAN K H. Distributed operation of wind farm for maximizing output power:A multi-agent deep reinforcement learning approach[J]. IEEE ACCESS, 2020, 8:173136-173146.
[17]	CHEN Y, MEI T, LIU W X, et al. Wind farm power generation control via double-network-based deep reinforcement learning[J]. IEEE Transactions on Industrial Informatics, 2021, 18(4):2321-2330.
[18]	JONKMAN J, BUTTERFIELD S, MUSIAL W, et al. Definition of a 5 MW reference wind turbine for offshore system development[R]. National Renewable Energy Lab, 2009.
[19]	BASTANKHAH M, PORTÉ-AGEL F. Experimental and theoretical study of wind turbine wakes in yawed conditions[J]. Journal of Fluid Mechanics, 2016, 806:506-541.
[20]	QIAN G W, ISHIHARA T. Wind farm power maximization through wake steering with a new multiple wake model for prediction of turbulence intensity[J]. Energy, 2021, 220:119680.
[21]	崔双双, 孙单勋. 分工况下风电机组各变量相关性研究[J]. 综合智慧能源, 2022, 44(12):49-55. doi: 10.3969/j.issn.2097-0706.2022.12.007
	CUI Shuangshuang, SUN Shanxun. Study on the correlation of wind turbine variables under different conditions[J]. Integrated Intelligent Energy, 2022, 44(12):49-55. doi: 10.3969/j.issn.2097-0706.2022.12.007
[22]	袁孝科, 沈石兰, 张茂松, 等. 基于可解释强化学习的智能虚拟电厂最优调度[J/OL]. 综合智慧能源,1-9(2024-09-29)[2024-10-06]. http://kns.cnki.net/kcms/detail/41.1461.TK.20240927.1628.002.html.
	YUAN Xiaoke, SHEN Shilan, ZHANG Maosong, et al. Optimal scheduling of intelligent virtual power plants based on explainable reinforcement learning[J/OL]. Integrated Intelligent Energy,1-9(2024-09-29)[2024-10-06]. http://kns.cnki.net/kcms/detail/41.1461.TK.20240927.1628.002.html.
[23]	Ministry of economic affairs netherlands enterprise agency and climate policy[Z]. Hollandse Kust Noord (Site b) Dataset, 2019. https://offshorewind.rvo.nl/file/view/55040229/Processed+data+HKNB.
[24]	NEUSTROEV G, Andringa S P E, VERZIJLBERGH R A, et al. Deep reinforcement learning for active wake control[C]// Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems. 2022: 944-953.

超参数	数值
Actor网络学习率	3×10^-4
Critic网络学习率	0.001
折扣系数	0.99
泛化优势估计	0.95
Actor，Critic网络更新轮次	10
裁剪参数	0.20

基于深度强化学习的风电场功率多变量综合优化控制

Multivariable integrated power control optimization of wind farms based on deep reinforcement learning

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 24

相关文章 15

编辑推荐

Metrics

本文评价

[1]	袁孝科, 沈石兰, 张茂松, 石晨旭, 杨凌霄. 基于可解释强化学习的智能虚拟电厂最优调度[J]. 综合智慧能源, 2025, 47(1): 1-9.
[2]	江阎, 宋晨辉, 张宁, 贺波, 卢宇婷. 考虑多能耦合的虚拟电厂优化运行模型与策略研究[J]. 综合智慧能源, 2025, 47(1): 34-41.
[3]	赵大周, 谢玉荣, 张钟平, 邓睿锋, 刘丽丽. 300 MW燃煤供热机组熔盐储热系统设计及经济性分析[J]. 综合智慧能源, 2024, 46(9): 45-52.
[4]	樊颜搏, 熊亚选, 李想, 田曦, 杨洋. 基于遗传算法的建筑用能多目标优化应用进展[J]. 综合智慧能源, 2024, 46(9): 69-85.
[5]	李菲菲, 徐绘薇, 崔金栋. 基于STIRPAT模型的吉林省石化行业碳排放影响因素研究[J]. 综合智慧能源, 2024, 46(8): 12-19.
[6]	李菲菲, 王舒泓, 崔金栋. 全生命周期视角下汽车产业碳排放影响因素的研究——以吉林省为例[J]. 综合智慧能源, 2024, 46(8): 20-27.
[7]	王泽宁, 李文中, 李东辉, 徐泰山, 俞俊. 基于软件定义的新型电力系统分层自治电力平衡模式研究[J]. 综合智慧能源, 2024, 46(7): 1-11.
[8]	何方波, 裴力耕, 郑睿, 范康健, 张晓曼, 李更丰. “源网荷储”协同助力陕西省新型电力系统建设[J]. 综合智慧能源, 2024, 46(7): 40-46.
[9]	王俊, 田浩, 赵二岗, 舒展, 万子镜. 计及电动汽车共享储能特性的园区柔性资源低碳运行控制方法[J]. 综合智慧能源, 2024, 46(6): 16-26.
[10]	郁海彬, 卢闻州, 唐亮, 张煜晨, 邹翔宇, 姜玉靓, 刘嘉宝. 考虑风险偏好的多主体虚拟电厂经济调度与收益分配策略[J]. 综合智慧能源, 2024, 46(6): 66-77.
[11]	王亮, 邓松. 面向新型电力系统的异常数据检测方法[J]. 综合智慧能源, 2024, 46(5): 12-19.
[12]	王永利, 王亚楠, 马子奔, 秦雨萌, 陈锡昌, 滕越. 面向区块链技术应用的能源交易系统效果评价[J]. 综合智慧能源, 2024, 46(4): 78-84.
[13]	孙健, 张云帆, 蔡潇龙, 刘鼎群. 基于预测负荷的暖通空调系统优化调度[J]. 综合智慧能源, 2024, 46(3): 12-19.
[14]	丁乐言, 柯松, 杨军, 施兴烨. 基于自适应控制参数整定的虚拟同步发电机控制策略[J]. 综合智慧能源, 2024, 46(3): 35-44.
[15]	苑曙光, 张瑜婷, 王峰, 苑广震. 蒙西地区规模化储能商业运行模式及风险分析[J]. 综合智慧能源, 2024, 46(3): 63-71.