Application and prospect of multimodal knowledge graph in electric power operation inspection

doi:10.3969/j.issn.2097-0706.2024.01.008

Abstract

Abstract:

In the context of building a new power system with new energy as the main body，knowledge graph（KG），a large-scale visual semantic network，is expanding its applications rapidly in power operation and inspection. The applications of KG in power operation and inspection mainly focus on semantic information processing. However，a large amount of heterogeneous data will be generated in power grid operation，being able to uphold the construction of multimodal knowledge graph（MMKG） which provides data support for various downstream tasks. In view of functional requirements on electric power inspection， MMKG is introduced to support the intelligent query answering system and fault handling. Expounding the construction technology of MMKG for power inspection data，the scenarios of power operation and inspection that MMKG can give full play in are summarized，and the development direction is forecasted. Finally，the challenges that will be faced by MMKG is analyzed comprehensively，which provides a reference for the development of intelligent power operation and inspection.

Key words: new power system, knowledge graph, multimodality, power operation and inspection, fault handling, artificial intelligence, big data

CLC Number:

TK01

LIN Jiajun, YAN Weidan, HU Junhua, ZHENG Yiming, SHAO Xianjun, GUO Bingyan. Application and prospect of multimodal knowledge graph in electric power operation inspection[J]. Integrated Intelligent Energy, 2024, 46(1): 65-74.

Figures/Tables 6

Table 1

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

References 50

[1]	刘平, 刘亮. 日本迈向碳中和的产业绿色发展战略——基于对《2050年实现碳中和的绿色成长战略》的考察[J]. 现代日本经济, 2021(4):14-27.
	LIU Ping, LIU Liang. Japan's green industrial development strategy towards carbon neutral:Based on the review of "Green Growth Strategy through Achieving Carbon Neutrality in 2050"[J]. Contemporary Economy of Japan, 2021(4):14-27.
[2]	喻小宝, 郑丹丹, 杨康, 等. “双碳”目标下能源电力行业的机遇与挑战[J]. 华电技术, 2021, 43(6):21-32.
	YU Xiaobao, ZHENG Dandan, YANG Kang, et al. Opportunities and challenges faced by energy and power industry with the goal of carbon neutrality and carbon peak[J]. Huadian Technology, 2021, 43(6):21-32.
[3]	张俊锋, 许文娟, 王跃锜, 等. 面向碳中和的中国碳排放现状调查与分析[J]. 华电技术, 2021, 43(10):1-10.
	ZHANG Junfeng, XU Wenjuan, WANG Yueqi, et al. Investigation and analysis on carbon emission status in China on the path to carbon neutrality[J]. Huadian Technology, 2021, 43(10):1-10.
[4]	索吉鑫, 李文娟, 杨生婧, 等. 电力运检突发事故的抢修及应对措施[J]. 电气时代, 2021(6):43-45.
	SUO Jixin, LI Wenjuan, YANG Shengjing, et al. Emergency repair and countermeasures for emergency accidents of electric power transportation inspection[J]. Electric Age, 2021(6):43-45.
[5]	郭榕, 杨群, 刘绍翰, 等. 电网故障处置知识图谱构建研究与应用[J]. 电网技术, 2021, 45(6):2092-2100.
	GUO Rong, YANG Qun, LIU Shaohan, et al. Construction and application of power grid fault handing knowledge graph[J]. Power System Technology, 2021, 45(6):2092-2100.
[6]	王琼, 杨波. 知识图谱在电力行业的应用与研究[J]. 网络安全技术与应用, 2020(11):137-138.
	WANG Qiong, YANG Bo. Application and research of knowledge graph in electric power industry[J]. Network Security Technology & Application, 2020(11):137-138.
[7]	肖发龙, 吴岳忠, 沈雪豪, 等. 基于深度学习和知识图谱的变电站设备故障智能诊断[J]. 电力建设, 2022, 43(3):66-74. doi: 10.12204/j.issn.1000-7229.2022.03.008
	XIAO Falong, WU Yuezhong, SHEN Xuehao, et al. Intelligent fault diagnosis of substation equipment on the basis of deep learning and knowledge graph[J]. Electric Power Construction, 2022, 43(3):66-74. doi: 10.12204/j.issn.1000-7229.2022.03.008
[8]	SZELISKI R. Computer vision: Algorithms and applications[M]. New York: Springer Nature, 2022.
[9]	BALTRUŠAITIS T, AHUJA C, MORENCY L P. Multimodal machine learning: A survey and taxonomy[J]. IEEE Trans on Pattern Analysis and Machine Intelligence, 2018, 41(2): 423-443. doi: 10.1109/TPAMI.2018.2798607
[10]	XIE R B, LIU Z Y, LUAN H B, et al. Image-embodied knowledge representation learning[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017: 3140-3146.
[11]	MOUSSELLY-SERGIEH H, BOTSCHEN T, GUREVYCH I, et al. A multimodal translation-based approach for knowledge graph representation learning[C]// Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics. 2018: 225-234.
[12]	AUER S, BIZER C, KOBILAROV G, et al. Dbpedia: A nucleus for a web of open data[M]. Berlin:Springer, 2007: 722-735.
[13]	VRANDEČIĆ D, KRÖTZSCH M. Wikidata: A free collaborative knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78-85.
[14]	FERRADA S, BUSTOS B, HOGAN A. IMGpedia: a linked dataset with content-based analysis of Wikimedia images[C]// International Semantic Web Conference. Springer, Cham, 2017: 84-93.
[15]	LIU Y, LI H, GARCIA-DURAN A, et al. MMKG: Multi-modal knowledge graphs[C]// European Semantic Web Conference. Springer, Cham, 2019: 459-474.
[16]	王萌, 王昊奋, 李博涵, 等. 新一代知识图谱关键技术综述[J]. 计算机研究与发展, 2022, 59(9):1947-1965.
	WANG Meng, WANG Haofen, LI Bohan, et al. Survey of key technologies of new generation knowledge graph[J]. Journal of Computer Research and Development, 2022, 59(9):1947-1965.
[17]	陈烨, 周刚, 卢记仓. 多模态知识图谱构建与应用研究综述[J]. 计算机应用研究, 2021, 38(12):3535-3543.
	CHEN Ye, ZHOU Gang, LU Jicang. Survey on construction and application research for multi-modal knowledge graphs[J]. Application Research of Computers, 2021, 38(12):3535-3543.
[18]	边晓燕, 张璐瑶, 周波, 等. 基于知识图谱的国内外电力市场研究综述[J]. 电工技术学报, 2022, 37(11):2777-2788.
	BIAN Xiaoyan, ZHANG Luyao, ZHOU Bo, et al. Review on domestic and international electricity market research based on knowledge graph[J]. Transactions of China Electrotechnical Society, 2022, 37(11):2777-2788.
[19]	李刚, 李银强, 王洪涛, 等. 电力设备健康管理知识图谱:基本概念、关键技术及研究进展[J]. 电力系统自动化, 2022, 46(3):1-13.
	LI Gang, LI Yinqiang, WANG Hongtao, et al. Knowledge graph of power equipment health management:Basic concepts,key technologies and research progress[J]. Automation of Electric Power Systems, 2022, 46(3):1-13.
[20]	张敏杰, 徐宁, 胡俊华, 等. 面向变压器智能运检的知识图谱构建和智能问答技术研究[J]. 全球能源互联网, 2020, 3(6):607-617.
	ZHANG Minjie, XU Ning, HU Junhua, et al. Knowledge graph construction and intelligent question answering for transformer operation and maintenance[J]. Journal of Global Energy Interconnection, 2020, 3(6):607-617.
[21]	赵振兵, 段记坤, 孔英会, 等. 基于门控图神经网络的栓母对知识图谱构建与应用[J]. 电网技术, 2021, 45(1):98-106.
	ZHAO Zhenbing, DUAN Jikun, KONG Yinghui, et al. Construction and application of bolt and nut pair knowledge graph based on GGNN[J]. Power System Technology, 2021, 45(1): 98-106.
[22]	刘梓权, 王慧芳. 基于知识图谱技术的电力设备缺陷记录检索方法[J]. 电力系统自动化, 2018, 42(14):158-164.
	LIU Ziquan, WANG Huifang. Retrieval method for defect records of power equipment based on knowledge graph technology[J]. Automation of Electric Power Systems, 2018, 42(14):158-164.
[23]	杨强. 基于知识图谱的核电设备知识平台建设[C]// 数字中国能源互联——2018电力行业信息化年会论文集, 2018:274-278.
[24]	汤亚宸, 方定江, 韩海韵, 等. 基于图数据库和知识图谱的电力设备质量综合管理系统研究[J]. 供用电, 2019, 36(11):35-40.
	TANG Yachen, FANG Dingjiang, HAN Haiyun, et al. Research on power equipment quality integrated management system based on graph database and knowledge graph[J]. Distribution & Utilization, 2019, 36(11):35-40.
[25]	赵振兵, 崔雅萍. 基于深度学习的输电线路关键部件视觉检测方法的研究进展[J]. 电力科学与工程, 2018, 34(3):1-6.
	ZHAO Zhenbing, CUI Yaping. Research progress of visual detection methods for transmission line key components based on deep learning[J]. Electric Power Science and Engineering, 2018, 34(3):1-6.
[26]	宋亚奇, 周国亮, 朱永利. 智能电网大数据处理技术现状与挑战[J]. 电网技术, 2013, 37(4):927-935.
	SONG Yaqi, ZHOU Guoliang, ZHU Yongli. Present status and challenges of big data processing in smart grid[J]. Power System Technology, 2013, 37(4):927-935
[27]	LEHTINEN J, MUNKBERG J, HASSELGREN J, et al. Noise2Noise: Learning image restoration without clean data[J]. arXiv preprint arXiv:1803.04189, 2018.
[28]	韩笑, 王新迎, 韩帅, 等. 基于不均衡数据集成学习的大型电力变压器状态评价方法[J]. 电网技术, 2021, 45(1):107-114.
	HAN Xiao, WANG Xinying, HAN Shuai, et al. Ensemble learning method for large-scale power transformer status evaluation based on imbalanced data[J]. Power System Technology, 2021, 45(1):107-114.
[29]	赵小凡, 李情. 基于元数据管理的半结构化数据安全转换方法[J]. 自动化仪表, 2022, 43(3):65-68,74.
	ZHAO Xiaofan, LI Qing. Secure conversion method of semi-structured data based on metadata management[J]. Process Automation Instrumentation, 2022, 43(3):65-68,74.
[30]	牛东晓, 斯琴卓娅, 王董禹, 等. 基于数据挖掘的电力装备企业多价值链协同数据预处理方法研究及应用[J/OL]. 中国管理科学:1-12(2022-09-01)[2023-02-01].https://doi.org/10.16381/j.cnki.issn1003-207x.2022.0286.
	NIU Dongxiao, SIQIN Zhuoya, WANG Dongyu, et al. Method and application of multi-value chain collaborative data mining in power equipment enterprises based on deep learning[J/OL]. Chinese Journal of Management Science:1-12(2022-09-01)[2023-02-01].https://doi.org/10.16381/j.cnki.issn1003-207x.2022.0286.
[31]	张吉祥, 张祥森, 武长旭, 等. 知识图谱构建技术综述[J]. 计算机工程, 2022, 48(3):23-37. doi: 10.19678/j.issn.1000-3428.0061803
	ZHANG Jixiang, ZHANG Xiangsen, WU Changxu, et al. Survey of knowledge graph construction techniques[J]. Computer Engineering, 2022, 48(3):23-37. doi: 10.19678/j.issn.1000-3428.0061803
[32]	林海伦, 王元卓, 贾岩涛, 等. 面向网络大数据的知识融合方法综述[J]. 计算机学报, 2017, 40(1):1-27.
	LIN Hailun, WANG Yuanzhuo, JIA Yantao, et al. Network big data oriented knowledge fusion methods:A survey[J]. Chinese Journal of Computers, 2017, 40(1):1-27.
[33]	MOON S, NEVES L, CARVALHO V. Multimodal named entity disambiguation for noisy social media posts[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018, 1: 2000-2008.
[34]	王会勇, 论兵, 张晓明, 等. 基于联合知识表示学习的多模态实体对齐[J]. 控制与决策, 2020, 35(12): 2855-2864.
	WANG Huiyong, LUN Bing, ZHANG Xiaoming, et al. Multi-modal entity alignment based on joint knowledge representation learning[J]. Control and Decision, 2020, 35(12):2855-2864.
[35]	王昊奋, 漆桂林, 陈华钧. 知识图谱:方法、实践与应用[M]. 北京: 电子工业出版社, 2019.
[36]	杨维, 孙德艳, 张晓慧, 等. 面向电力智能问答系统的命名实体识别算法[J]. 计算机工程与设计, 2019, 40(12):3625-3630.
	YANG Wei, SUN Deyan, ZHANG Xiaohui, et al. Named entity recognition for intelligent answer system in power service[J]. Computer Engineering and Design, 2019, 40(12):3625-3630.
[37]	王秋琳, 梁懿, 闫丽飞, 等. 基于电力智能问答系统的命名实体识别算法研究[J]. 电子技术与软件工程, 2022(3):95-98.
	WANG Qiulin, LIANG Yi, YAN Lifei, et al. Research on named entity recognition algorithm based on power intelligent question answering system[J]. Electronic Technology & Software Engineering, 2022(3):95-98.
[38]	杨志明, 张征容, 易亮, 等. 基于智能辨识的配电网运检知识服务应用模型[J]. 云南电力技术, 2021, 49(2):45-50.
	YANG Zhiming, ZHANG Zhengrong, YI Liang, et al. Application model of knowledge service for distribution network transportation inspection based on intelligent identifcation[J]. Yunnan Electric Power, 2021, 49(2):45-50.
[39]	盛慧, 任堂正, 包云康, 等. 基于知识图谱和语音识别的电网运检智能AI助手[J]. 电子技术与软件工程, 2021(20):152-153.
	SHENG Hui, REN Tangzheng, BAO Yunkang, et al. Intelligent AI assistant for power grid inspection based on knowledge graph and voice recognition[J]. Electronic Technology & Software Engineering, 2021(20):152-153.
[40]	徐会芳, 张中浩, 谈元鹏, 等. 面向电网调度领域的实体识别技术[J]. 电力建设, 2021, 42(10):71-77. doi: 10.12204/j.issn.1000-7229.2021.10.008
	XU Huifang, ZHANG Zhonghao, TAN Yuanpeng, et al. Research on entity recognition technology in power grid dispatching field[J]. Electric Power Construction, 2021, 42(10):71-77. doi: 10.12204/j.issn.1000-7229.2021.10.008
[41]	高欣, 任昺, 张浩, 等. 基于信息差异图模型的电力调度自动化系统组件故障溯源方法[J]. 电网技术, 2021, 45(12):4808-4817.
	GAO Xin, REN Bing, ZHANG Hao, et al. Component fault tracing of power dispatching automation system based on information difference graph model[J]. Power System Technology, 2021, 45(12):4808-4817.
[42]	乔骥, 王新迎, 闵睿, 等. 面向电网调度故障处理的知识图谱框架与关键技术初探[J]. 中国电机工程学报, 2020, 40(18):5837-5848.
	QIAO Ji, WANG Xinying, MIN Rui, et al. Framework and key technologies of knowledge-graph-based fault handling system in power grid[J]. Proceedings of the CSEE, 2020, 40(18):5837-5848.
[43]	戴宇欣, 张俊, 季知祥, 等. 基于功能缺陷文本的电力系统二次设备智能诊断与辅助决策[J]. 电力自动化设备, 2021, 41(6):184-191.
	DAI Yuxin, ZHANG Jun, JI Zhixiang, et al. Intelligent diagnosis and auxiliary decision of power system secondary equipment based on functional defect text[J]. Electric Power Automation Equipment, 2021, 41(6):184-191.
[44]	朱孟周, 孙志明, 黄强, 等. 基于Gabor变换的电力计量设备运行故障诊断技术[J]. 自动化与仪器仪表, 2021(10): 249-252.
	ZHU Mengzhou, SUN Zhiming, HUANG Qiang, et al. Fault diagnosis technology of power metering equipment based on Gabor transform[J]. Automation & Instrumentation, 2021(10): 249-252.
[45]	郭方洪, 刘师硕, 吴祥, 等. 基于联邦学习的含不平衡样本数据电力变压器故障诊断[J/OL]. 电力系统自动化:1-10(2022-08-30)[2023-02-01]. https://kns.cnki.net/kcms/detail/32.1180.TP.20220829.0958.002.html.
	GUO Fanghong, LIU Shishuo, WU Xiang, et al. Fault diagnosis of power transformers with unbalanced sample data based on federated learning[J/OL]. Automation of Electric Power Systems:1-10(2022-08-30)[2023-02-01]. https://kns.cnki.net/kcms/detail/32.1180.TP.20220829.0958.002.html.
[46]	PALMONARI M, MINERVINI P. Knowledge graph embeddings and explainable AI[J]. Knowledge Graphs for Explainable Artificial Intelligence :Foundations, Applications and Challenges, 2020, 47: 49.
[47]	王骏东, 杨军, 裴洋舟, 等. 基于知识图谱的配电网故障辅助决策研究[J]. 电网技术, 2021, 45(6):2101-2112.
	WANG Jundong, YANG Jun, PEI Yangzhou, et al. Distribution network fault assistant decision-making based on knowledge graph[J]. Power System Technology, 2021, 45(6):2101-2112.
[48]	蒲天骄, 谈元鹏, 彭国政, 等. 电力领域知识图谱的构建与应用[J]. 电网技术, 2021, 45(6):2080-2091.
	PU Tianjiao, TAN Yuanpeng, PENG Guozheng, et al. Construction and application of knowledge graph in the electric power field[J]. Power System Technology, 2021, 45(6):2080-2091.
[49]	陈峥, 尹耕, 康晓东, 等. 基于大数据平台的电力设备运作状态监测系统研究[J]. 电力大数据, 2019, 22(11):1-7.
	CHEN Wei, YIN Geng, KANG Xiaodong, et al. Research on monitoring system of power equipment operation status based on big data platform[J]. Power Systems and Big Data, 2019, 22(11):1-7.
[50]	PAREJA A, DOMENICONI G, CHEN J, et al. EvolveGCN: Evolving graph convolutional networks for dynamic graphs[C]// Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 34(4), 5363-5370.

KG名称	数据规模	适用领域
Freebase	6 800万个实例，24亿个三元组	通用
Dbpedia	40亿个实体，2 800万个知识联系，95亿个三元组	通用
YAGO	980万个实体，114个知识联系，4.5亿个三元组	通用
Zhishi.me	超1 200万个实例，超1.2亿个三元组	通用
XLORE	66万个实体，5万个属性，1 000万个实例	通用
Google Knowledge Graph	5.8亿个实体，3.5万个知识联系，180亿个三元组	通用
开源军事武器装备KG	武器类实体5 800个，实体属性关系184类，实体上位关系1类，包括8大类、148小类的武器装备，涉及国家88个	军事
Linked Movie Dataset	61.5万个三元组数据	影视
乳腺癌KG	超过2 200万个三元组	医学