Optimal power flow calculation of power grid based on reinforcement learning and crisscross PSO algorithm particle swarm optimization

doi:10.3969/j.issn.1674-1951.2021.08.011

Abstract

Abstract:

To solve the optimal power flow in power systems,new particle swarm optimization（PSO）algorithm based on Q learning and crisscross search is proposed.The improved algorithm introduces crossover operator into PSO mode to enhance the global convergence ability.At the same time,introducing the exploration mode of Q learning into the improved algorithm makes the algorithm explore in the known solution space,so as to better balance the relationship between exploration and utilization.In order to solve the dimension disaster of Q learning algorithm,the method of state-action chain is used.Simulation results of IEEE57 and IEEE118 node systems show that the proposed algorithm can enhance the global convergence of the traditional PSO algorithm,and effectively solve large-scale optimal power flow problems.

Key words: optimal power flow, improved particle swarm, Q learning, crisscross algorithm, swarm intelligence optimization

CLC Number:

TM744

MENG Anbo, WANG Peng, DING Weifeng, CHEN Shun, LIANG Ruduo, ZHANG Zheng. Optimal power flow calculation of power grid based on reinforcement learning and crisscross PSO algorithm particle swarm optimization[J]. Huadian Technology, 2021, 43(8): 74-82.

Figures/Tables 9

Fig.1

Tab.1

Tab.2

Optimal setting of control variables in case 1

控制变量	CS-QPSO	控制变量	CS-QPSO
$P G 2$ /MW	89.25	$T 24 - 25$ /p.u.	0.99
$P G 3$ /MW	44.03	$T 24 - 26$ /p.u.	1.01
$P G 6$ /MW	69.16	$T 7 - 29$ /p.u.	0.92
$P G 8$ /MW	460.70	$T 32 - 34$ /p.u.	0.95
$P G 9$ /MW	100.00	$T 11 - 41$ /p.u.	0.95
$P G 12$ /MW	357.93	$T 15 - 45$ /p.u.	0.91
$V 1$ /p.u.	1.07	$T 14 - 46$ /p.u.	0.90
$V 2$ /p.u.	1.07	$T 10 - 51$ /p.u.	0.92
$V 3$ /p.u.	1.07	$T 13 - 49$ /p.u.	0.90
$V 6$ /p.u.	1.08	$T 11 - 43$ /p.u.	0.90
$V 8$ /p.u.	1.10	$T 40 - 56$ /p.u.	1.10
$V 9$ /p.u.	1.08	$T 39 - 57$ /p.u.	1.01
$V 12$ /p.u.	1.06	$T 9 - 55$ /p.u.	0.94
$T 4 - 18$ /p.u.	1.00	$Q C 18$ /Mvar	6.44
$T 4 - 18$ /p.u.	1.01	$Q C 25$ /Mvar	11.99
$T 21 - 20$ /p.u.	1.10	$Q C 53$ /Mvar	11.06
$T 24 - 25$ /p.u.	1.01

Tab.2

Fig.2

Tab.3

Tab.4

Optimal setting of control variables in case 2

控制变量	CS-QPSO	控制变量	CS-QPSO	控制变量	CS-QPSO
$P G 2$ /MW	0.00	$P G 46$ /MW	38.33	$V G 36$ /p.u.	1.04
$P G 3$ /MW	2.84	$P G 47$ /MW	0.47	$V G 37$ /p.u.	1.05
$P G 4$ /MW	0.00	$P G 48$ /MW	7.10	$V G 38$ /p.u.	1.05
$P G 5$ /MW	406.68	$P G 49$ /MW	29.08	$V G 39$ /p.u.	1.04
$P G 6$ /MW	86.46	$P G 50$ /MW	8.03	$V G 40$ /p.u.	1.05
$P G 7$ /MW	19.90	$P G 51$ /MW	35.18	$V G 41$ /p.u.	1.03
$P G 8$ /MW	16.22	$P G 52$ /MW	35.22	$V G 42$ /p.u.	1.03
$P G 9$ /MW	18.72	$P G 53$ /MW	0.00	$V G 43$ /p.u.	1.04
$P G 10$ MW	0.00	$P G 54$ /MW	0.00	$V G 44$ /p.u.	1.04
$P G 11$ /MW	196.11	$V G 1$ /p.u.	1.03	$V G 45$ /p.u.	1.04
$P G 12$ /MW	282.58	$V G 2$ /p.u.	1.05	$V G 46$ /p.u.	1.02
$P G 13$ /MW	13.20	$V G 3$ /p.u.	1.04	$V G 47$ /p.u.	1.02
$P G 14$ /MW	7.23	$V G 4$ /p.u.	1.10	$V G 48$ /p.u.	1.01
$P G 15$ /MW	13.92	$V G 5$ /p.u.	1.10	$V G 49$ /p.u.	1.00
$P G 16$ /MW	3.88	$V G 6$ /p.u.	1.04	$V G 50$ /p.u.	1.01
$P G 17$ /MW	11.87	$V G 7$ /p.u.	1.03	$V G 51$ /p.u.	1.02
$P G 18$ /MW	49.38	$V G 8$ /p.u.	1.04	$V G 52$ /p.u.	1.00
$P G 19$ /MW	42.55	$V G 9$ /p.u.	1.03	$V G 53$ /p.u.	1.04
$P G 20$ /MW	19.24	$V G 10$ /p.u.	1.05	$V G 54$ /p.u.	1.10
$P G 21$ /MW	193.26	$V G 11$ /p.u.	1.08	$T 1$ /p.u.	1.03
$P G 22$ /MW	49.33	$V G 12$ /p.u.	1.10	$T 2$ /p.u.	1.10
$P G 23$ /MW	29.48	$V G 13$ /p.u.	1.05	$T 3$ /p.u.	1.04
$P G 24$ /MW	31.42	$V G 14$ /p.u.	1.03	$T 4$ /p.u.	1.04
$P G 25$ /MW	149.66	$V G 15$ /p.u.	1.04	$T 5$ /p.u.	1.03
$P G 26$ /MW	146.83	$V G 16$ /p.u.	1.04	$T 6$ /p.u.	1.03
$P G 27$ /MW	0.00	$V G 17$ /p.u.	1.04	$T 7$ /p.u.	1.02
$P G 28$ /MW	354.45	$V G 18$ /p.u.	1.03	$T 8$ /p.u.	0.97
$P G 29$ /MW	349.49	$V G 19$ /p.u.	1.04	$T 9$ /p.u.	1.03
$P G 30$ /MW	706.21	$V G 20$ /p.u.	1.05	$Q C 1$ /MVar	13.10
$P G 31$ /MW	0.00	$V G 21$ /p.u.	1.06	$Q C 2$ /MVar	3.83
$P G 32$ /MW	0.00	$V G 22$ /p.u.	1.04	$Q C 3$ /MVar	0.00
$P G 33$ /MW	0.00	$V G 23$ /p.u.	1.03	$Q C 4$ /MVar	5.13
$P G 34$ /MW	19.42	$V G 24$ /p.u.	1.03	$Q C 5$ /MVar	19.14
$P G 35$ /MW	22.67	$V G 25$ /p.u.	1.06	$Q C 6$ /MVar	21.77
$P G 36$ /MW	0.00	$V G 26$ /p.u.	1.06	$Q C 7$ /MVar	12.68
$P G 37$ /MW	432.87	$V G 27$ /p.u.	1.06	$Q C 8$ /MVar	24.17
$P G 38$ /MW	0.00	$V G 28$ /p.u.	1.10	$Q C 9$ /MVar	30.00
$P G 39$ /MW	3.59	$V G 29$ /p.u.	1.08	$Q C 10$ /MVar	16.11
$P G 40$ /MW	503.14	$V G 30$ /p.u.	1.08	$Q C 11$ /MVar	16.30
$P G 41$ /MW	0.00	$V G 31$ /p.u.	1.05	$C 12$ /MVar	11.02
$P G 42$ /MW	0.00	$V G 32$ /p.u.	1.04	$C 13$ /MVar	30.00
$P G 43$ /MW	0.00	$V G 33$ /p.u.	1.04	$C 14$ /MVar	17.35
$P G 44$ /MW	0.00	$V G 34$ /p.u.	1.04
$P G 45$ /MW	230.86	$V G 35$ /p.u.	1.03

Tab.4

Fig.3

Tab.5

Fig.4

References 27

[1]	李春晓, 何仁君. 基于内点法的最优潮流计算及算例分析[J]. 电气开关, 2018, 56(1):32-36.
	LI Chunxiao, HE Renjun. The calculation of optimal power flow based on interior-point method[J]. Electric Switchgear, 2018, 56(1):32-36.
[2]	张江红, 孟宪朋, 刘怀东, 等. 最优潮流算法综述[J]. 华北电力技术, 2010(7):23-26,45.
	ZHANG Jianghong, MENG Xianpeng, LIU Huaidong, et al. A summary of algorithms for optimal power flow[J]. North China Electric Power, 2010(7):23-26,45.
[3]	李彩华, 郭志忠. 最优潮流的发展[J]. 继电器, 2002(1):1-6,56.
	LI Caihua, GUO Zhizhong. The development of optimal power flow[J]. Relay, 2002(1):1-6,56.
[4]	陈丽光, 文波, 聂一雄. 基于退火粒子群和内点法的改进最优潮流算法[J]. 广东电力, 2013, 26(9):32-35,103.
	CHEN Liguang, WEN Bo, NIE Yixiong. Improved optimal power flow algorithm based on annealing particle swarm and interior point method[J]. Guangdong Electric Power, 2013, 26(9):32-35,103.
[5]	刘敏. 基于改进遗传算法的最优潮流问题的研究[D]. 长沙:中南大学, 2010.
[6]	肖儿良, 林蔚, 毛海军, 等. 基于权值的引力搜索算法在电力系统最优潮流计算中的应用[J]. 电工电能新技术, 2014, 33(7):62-66.
	XIAO Erliang, LIN Wei, MAO Haijun, et al. Calculation of optimal power flow problem using search algorithm based on weight of gravitation[J]. Advanced Technology of Electrical Engineering and Energy, 2014, 33(7):62-66.
[7]	陈维荣, 张倩, 王劲草, 等. 搜寻者优化算法在最优潮流中的应用[J]. 电力系统及其自动化学报, 2009, 21(1):64-67.
	CHEN Weirong, ZHANG Qian, WANG Jincao, et al. Application of seeker optimization algorithm in optimal power flow[J]. Proceedings of the CSU-EPSA, 2009, 21(1):64-67.
[8]	NGUYEN T T. A high performance social spider optimization algorithm for optimal power flow solution with single objective optimization[J]. Energy, 2019, 171:218-240. doi: 10.1016/j.energy.2019.01.021
[9]	赵鑫, 郑文禹, 侯智华, 等. 基于粒子群优化算法的多能互补系统经济调度研究[J]. 华电技术, 2021, 43(4):14-20.
	ZHAO Xin, ZHENG Wenyu, HOU Zhihua, et al. Research on economic dispatch of multi-energy complementary system based on Particle Swarm Optimization[J]. Huadian Technology, 2021, 43(4):14-20.
[10]	陆立民, 褚国伟, 张涛, 等. 基于改进多目标粒子群算法的微电网储能优化配置[J]. 电力系统保护与控制, 2020, 48(15):116-124.
	LU Limin, CHU Guowei, ZHANG Tao, et al. Optimal configuration of energy storage in a microgrid based on improved multi-objective particle swarm optimization[J]. Power System Protection and Control, 2020, 48(15):116-124.
[11]	李彬, 郝一浩, 祁兵, 等. 边缘计算在需求响应中的应用现状及发展思路[J]. 华电技术, 2021, 43(3):1-8.
	LI Bin, HAO Yihao, QI Bing, et al. Application and development of edge computing in demand response[J]. Huadian Technology, 2021, 43(3):1-8.
[12]	吴攀. 光伏发电系统发电功率预测[J]. 发电技术, 2020, 41(3):231-236.
	WU Pan. Power forecasting of photovoltaic power generation system[J]. Power Generation Technology, 2020, 41(3):231-236.
[13]	高明明, 杨磊, 于浩洋, 等. 量子计算在火电机组优化控制中的应用综述[J]. 华电技术, 2020, 42(8):90-96.
	GAO Mingming, YANG Lei, YU Haoyang, et al. Review on the application of quantum computing in optimization control on thermal power units[J]. Huadian Technology, 2020, 42(8):90-96.
[14]	谈智玲, 陈才明, 徐胜朝, 等. 基于振动信号分析的滚动轴承寿命预测方法研究[J]. 华电技术, 2021, 43(5):36-44.
	TAN Zhiling, CHEN Caiming, XU Shengchao, et al. Research on service life prediction on rolling bearings based on vibration signal analysis[J]. Huadian Technology, 2021, 43(5):36-44.
[15]	李鑫滨, 朱庆军, 马红霞, 等. 粒子群算法及其在电力系统无功优化中的应用综述[J]. 燕山大学学报, 2008(3):245-250.
	LI Xinbin, ZHU Qingjun, MA Hongxia, et al. Overview of particle swarm optimization algorithm on reactive optimization for power system[J]. Journal of Yanshan University, 2008(3):245-250.
[16]	MENG A B, CHEN Y C, YIN H, et al. Crisscross optimization algorithm and its application[J]. Knowledge-Based, 2014, 67:218-229.
[17]	胡细兵. 基于强化学习算法的最优潮流研究[D]. 广州:华南理工大学, 2011.
[18]	NARIMANI M R, AZIZIPANAH-ABARGHOOEE R, ZOGHDAR-MOGHADAM-SHAHREKOHNE B, et al. A novel approach to multi-objective optimal power flow by a new hybrid optimization algorithm considering generator constraints and multi-fuel type[J]. Energy, 2013, 49:119-136. doi: 10.1016/j.energy.2012.09.031
[19]	NADERI E, NARIMANI H, FATHI M, et al. Narimani a novel fuzzy adaptive configuration of particle swarm optimization to solve large-scale optimal reactive power dispatch[J]. Applied Soft Computing, 2017, 53:441-456. doi: 10.1016/j.asoc.2017.01.012
[20]	NADERI E, POURAKBARI-KASMAEI M, CERNA F V, et al. A novel hybrid self-adaptive heuristic algorithm to handle single-and multi-objective optimal power flow problems[J]. International Journal of Electrical Power & Energy Systems, 2021, 125.DOI: 10.1016/j.ijepes.2020.106492. doi: 10.1016/j.ijepes.2020.106492
[21]	SHILAJA C, ARUNPRASATH T. Optimal power flow using moth swarm algorithm[J]. Future Generation Computer Systems, 2019, 98:708-715. doi: 10.1016/j.future.2018.12.046
[22]	DUMAN S, GÜVENÇ U, SÖNMEZ Y, et al. Optimal power flow using gravitational search algorithm[J]. Energy Conversion and Management, 2012, 59:86-95. doi: 10.1016/j.enconman.2012.02.024
[23]	GHASEMI M, GHAVIDEL S, GHANBARIAN M M, et al. Application of imperialist competitive algorithm with its modified techniques for multi-objective optimal power flow problem:A comparative study[J]. Information Sciences, 2014, 281:225-247. doi: 10.1016/j.ins.2014.05.040
[24]	HMIDA J B, CHAMBERS T, LEE J. Solving constrained optimal power flow with renewables using hybrid modified imperialist competitive algorithm and sequential quadratic programming[J]. Electric Power Systems Research, 2019, 177.DOI: 10.1016/j.epsr.2019.105989. doi: 10.1016/j.epsr.2019.105989
[25]	SINGH R P, MUKHERJEE V, GHOSHAL S P. Particle swarm optimization with an aging leader and challengers algorithm for the solution of optimal power flow problem[J]. Applied Soft Computing, 2016, 40:161-177. doi: 10.1016/j.asoc.2015.11.027
[26]	ROBERGE V, TARBOUCHI M, OKOU F. Optimal power flow based on parallel metaheuristics for graphics processing units[J]. Electric Power Systems Research 2016, 140:344-353. doi: 10.1016/j.epsr.2016.06.006
[27]	BAI W, EKE I, LEE K Y. An improved artificial bee colony optimization algorithm based on orthogonal learning for optimal power flow problem[J]. Control Engineering Practice, 2017, 61:163-172. doi: 10.1016/j.conengprac.2017.02.010

优化算法	最小值	平均值	最大值
HMICA-SQP	41 881.66	—	—
GA	41 711.94	41 719.60	41 734.16
DE	41 709.88	41 715.29	41 720.36
GSA	41 695.87	—	—
MSA	41 673.72	—	—
NISSO	41 665.54	—	—
FAHSPSO-DE	41 637.18	—	—
PSO	41 670.62	41 745.37	41 837.77
CS-QPSO	41 590.86	41 604.08	41 621.14

优化算法	最小值	平均值	最大值
SSO	132 080.41	—	—
PSO	131 005.73	—	—
PSO	130 062.86	130 142.48	130 234.34
GPU-PSO	129 627.03	—	—
NISSO	129 879.45	—	—
IABC	129 862.00	129 895.00	129 941.00
MSA	129 640.72	—	—
PSO	130 344.33	131 417.60	133 016.09
CS-QPSO	129 568.55	129 627.23	129 698.87

优化算法	最小值	平均值	最大值
NISSO	9.98	—	—
FAHSPSO-DE	11.73	—	—
SSO	10.61	—	—
PSO	9.25	9.77	10.60
CS-QPSO	8.79	9.20	9.70