Multivariable integrated power control optimization of wind farms based on deep reinforcement learning

doi:10.3969/j.issn.2097-0706.2025.01.003

Abstract

Abstract:

China has proposed the"dual carbon"strategy，aiming to build a new power system with renewable energy as the primary component.Based on real wind farm data，optimization control strategies were proposed to improve the wind farm's output power，thereby further enhancing wind energy utilization.The wake effects between wind turbines were the main focus，and a wind farm power multivariable optimization control strategy based on model-free deep reinforcement learning（DRL）was proposed.The strategy employed the Proximal Policy Optimization（PPO）algorithm to optimize multiple variables，including the yaw angle，tilt angle，blade pitch angle，and tip speed ratio（TSR） in a dynamic wind farm.Through intelligent agents learning from the data generated by the agent during operation，an optimal control strategy was obtained，overcoming the limitations of traditional mathematical optimization methods. Simulation results showed that，compared to existing wind turbine control algorithms，the model-free DRL-based multivariable optimization control strategy significantly improved computational efficiency，reduced the difficulty of parameter optimization，and optimized the direction and strength of the wake.The optimized average output power was increased by 37.08%.

Key words: wind farm, deep reinforcement learning, wake control, multivariable control, power optimization, PPO, "dual carbon" strategy, new power system

CLC Number:

TK81

ZHANG Huaqin, LIU Wei, WANG Hui, LI Leixiao, Sharengaowa. Multivariable integrated power control optimization of wind farms based on deep reinforcement learning[J]. Integrated Intelligent Energy, 2025, 47(1): 18-25.

Figures/Tables 12

Fig.1

Fig.2

Fig.3

Table 1

Wind farm parameters

参数	数值
风电机组直径 $/ m$	126
风电机组之间的距离 $/$ m	630
轮毂高度 $/ m$	90
空气密度 $/ (k g · m - 3)$	1.225
切入风速 $/ (m · s - 1)$	3
切出风速 $/ (m · s - 1)$	25

Table 1

Fig.4

Table 2

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

References 24

[1]	朱博, 吴水军, 黄柯昊, 等. 水风光互补系统一次调频方案研究[J]. 太阳能学报, 2024, 45(5):412-421.
	ZHU Bo, WU Shuijun, HUANG Kehao, et al. Research on primary frequency regulation scheme of water-wind-solar complementary system[J]. Acta Energiae Solaris Sinica, 2024, 45(5):412-421.
[2]	瞿国华. 我国能源转型与过渡能源的合理选择[J]. 科学发展, 2022(12):88-96.
	QU Guohua. China's rational choice of energy transition and transition energy[J]. Scientific Development, 2022(12):88-96.
[3]	KUMAR D, CHATTERJEE K. A review of conventional and advanced MPPT algorithms for wind energy systems[J]. Renewable and Sustainable Energy Reviews, 2016, 55:957-970.
[4]	宋冬然, 沈旭涛, 黄朝能, 等. 基于代理模型辅助改进标准粒子群算法的浮式风电场功率优化[J]. 中国电机工程学报, 2023, 43(S1):217-228.
	SONG Dongran, SHEN Xutao, HUANG Chaoneng, et al. Power optimization of floating wind farm based on improved standard particle swarm optimization assisted by proxy model[J]. Proceedings of the CSEE, 2023, 43(S1):217-228.
[5]	NASH R, NOURI R, VASEL A. Wind turbine wake control strategies:A review and concept proposal[J]. Energy Conversion and Management, 2021, 245:114581.
[6]	CAO L Y, WEI W, ZHANG J W, et al. Adaptive economic predictive control for offshore wind farm active yaw considering generation uncertainty[J]. Applied Energy, 2023, 351:121849.
[7]	李雄威, 徐家豪, 朱润泽, 等. 基于偏航尾流模型的风电场功率协同优化研究[J]. 太阳能学报, 2022, 43(10):144-151. doi: 10.19912/j.0254-0096.tynxb.2021-1292
	LI Xiongwei, XU Jiahao, ZHU Runze, et al. Study on power collaborative optimization of wind farm based on yaw wake model[J]. Acta Energiae Solaris Sinica, 2022, 43(10):144-151. doi: 10.19912/j.0254-0096.tynxb.2021-1292
[8]	SUN X X, LI Y M, ZHANG F, et al. Decentralized yaw optimization for maximizing wind farm production based on deep reinforcement learning[J]. Energy Conversion and Management, 2023, 286:1-11.
[9]	LIU Y, ZHANG J H, ZHOU J H. The influence of tilt angle on the aerodynamic performance of a wind turbine[J]. Applied Sciences, 2020, 10(15):5380.
[10]	XU X H, LIU D F, LI Y, et al. Collective large-scale wind farm multivariate power output control based on hierarchical communication multi-agent proximal policy optimization[J]. Renewable Energy, 2023, 219:119479.
[11]	SUTTON R S, BARTO A G. Reinforcement learning:An introduction[M]. MIT Press, 2018.
[12]	LI Y. Deep reinforcement learning:An overview[Z]. 2017.
[13]	王康平, 张兴科, 刘财华, 等. 基于自适应下垂控制的风电场无功电压控制策略[J]. 综合智慧能源, 2022, 44(4):12-19. doi: 10.3969/j.issn.2097-0706.2022.04.002
	WANG Kangping, ZHANG Xingke, LIU Caihua, et al. Reactive power and voltage control strategy based on adaptive droop control for wind power plants[J]. Integrated Intelligent Energy, 2022, 44(4):12-19. doi: 10.3969/j.issn.2097-0706.2022.04.002
[14]	刘玉山, 胡阔海, 王灵梅, 等. 考虑尾流效应的风电场输出功率优化[J]. 可再生能源, 2023, 41(10):1336-1342.
	LIU Yushan, HU Kuohai, WANG Lingmei, et al. Optimization of wind farm output power considering wake effect[J]. Renewable Energy Resources, 2023, 41(10):1336-1342.
[15]	PADULLAPARTHI V R, NAGARATHINAM S, VASAN A, et al. Falcon-farm Level control for wind turbines using multi-agent deep reinforcement learning[J]. Renewable Energy, 2022, 181:445-456.
[16]	HAI B V, THANH N T, MAN K H. Distributed operation of wind farm for maximizing output power:A multi-agent deep reinforcement learning approach[J]. IEEE ACCESS, 2020, 8:173136-173146.
[17]	CHEN Y, MEI T, LIU W X, et al. Wind farm power generation control via double-network-based deep reinforcement learning[J]. IEEE Transactions on Industrial Informatics, 2021, 18(4):2321-2330.
[18]	JONKMAN J, BUTTERFIELD S, MUSIAL W, et al. Definition of a 5 MW reference wind turbine for offshore system development[R]. National Renewable Energy Lab, 2009.
[19]	BASTANKHAH M, PORTÉ-AGEL F. Experimental and theoretical study of wind turbine wakes in yawed conditions[J]. Journal of Fluid Mechanics, 2016, 806:506-541.
[20]	QIAN G W, ISHIHARA T. Wind farm power maximization through wake steering with a new multiple wake model for prediction of turbulence intensity[J]. Energy, 2021, 220:119680.
[21]	崔双双, 孙单勋. 分工况下风电机组各变量相关性研究[J]. 综合智慧能源, 2022, 44(12):49-55. doi: 10.3969/j.issn.2097-0706.2022.12.007
	CUI Shuangshuang, SUN Shanxun. Study on the correlation of wind turbine variables under different conditions[J]. Integrated Intelligent Energy, 2022, 44(12):49-55. doi: 10.3969/j.issn.2097-0706.2022.12.007
[22]	袁孝科, 沈石兰, 张茂松, 等. 基于可解释强化学习的智能虚拟电厂最优调度[J/OL]. 综合智慧能源,1-9(2024-09-29)[2024-10-06]. http://kns.cnki.net/kcms/detail/41.1461.TK.20240927.1628.002.html.
	YUAN Xiaoke, SHEN Shilan, ZHANG Maosong, et al. Optimal scheduling of intelligent virtual power plants based on explainable reinforcement learning[J/OL]. Integrated Intelligent Energy,1-9(2024-09-29)[2024-10-06]. http://kns.cnki.net/kcms/detail/41.1461.TK.20240927.1628.002.html.
[23]	Ministry of economic affairs netherlands enterprise agency and climate policy[Z]. Hollandse Kust Noord (Site b) Dataset, 2019. https://offshorewind.rvo.nl/file/view/55040229/Processed+data+HKNB.
[24]	NEUSTROEV G, Andringa S P E, VERZIJLBERGH R A, et al. Deep reinforcement learning for active wake control[C]// Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems. 2022: 944-953.

超参数	数值
Actor网络学习率	3×10^-4
Critic网络学习率	0.001
折扣系数	0.99
泛化优势估计	0.95
Actor，Critic网络更新轮次	10
裁剪参数	0.20