Detection and repair of abnormal load data of public buildings based on MDLOF-iForest and M-KNN-Slope

doi:10.3969/j.issn.2097-0706.2025.03.006

Abstract

Abstract:

In research on energy consumption of public buildings， making the detection and repair of abnormal load data an indispensable part of data processing. To address the limitations of existing methods， a method based on the Mahalanobis distance-based local outlier factor-isolation forest （MDLOF-iForest） algorithm and the modified K-nearest neighbors-slope （M-KNN-Slope） algorithm was proposed.The MDLOF-iForest algorithm incorporated Mahalanobis distance into the traditional local outlier factor algorithm， improving the models ability to perceive correlations between data features. Meanwhile， by combining the advantages of MDLOF algorithm and iForest algorithm， it enabled rapid and accurate detection of abnormal data. The M-KNN-Slope algorithm used neighbors with similar load trend line characteristics of abnormal data and normal data to obtain the weighted average values of similar trend line slopes， completing the repair of abnormal data and reducing reliance on sample data. Verification was conducted using load data from an office public building and a commercial public building in Nanning， from August to November 2024. The results showed that approximately 90% of the repaired data had a difference of less than 10% compared to the correct data. Compared with conventional algorithms，the M-KNN-Slope algorithm could obtain more data with errors within 5%. Extreme gradient boosting， long short-term memory network， backpropagation neural network， and support vector machine were used to predict the data before and after repair. The root mean square values decreased by 5.02% to 17.83%， and the absolute mean errors decreased by 2.44% to 13.34%.

Key words: energy consumption of public buildings, load dataset, abnormal data detection, abnormal data repair, Mahalanobis distance-based local outlier factor-isolation forest, modified K-nearest neighbors-slope

CLC Number:

TK01

LIU Yining, CHEN Baian, DU Pengcheng, LIN Xiaogang, JIANG Meihui. Detection and repair of abnormal load data of public buildings based on MDLOF-iForest and M-KNN-Slope[J]. Integrated Intelligent Energy, 2025, 47(3): 62-72.

Figures/Tables 13

Table 1

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 2

Fig.6

Fig.7

Fig.8

Fig.9

Table 3

Table 4

References 27

[1]	中国建筑节能协会, 重庆大学城乡建设与发展研究院. 中国建筑能耗与碳排放研究报告(2023年)[J]. 建筑, 2024(2): 46-59.
[2]	孙健, 张云帆, 蔡潇龙, 等. 基于预测负荷的暖通空调系统优化调度[J]. 综合智慧能源, 2024, 46(3): 12-19. doi: 10.3969/j.issn.2097-0706.2024.03.002
	SUN Jian, ZHANG Yunfan, CAI Xiaolong, et al. Optimal scheduling of HVAC systems based on predicted loads[J]. Integrated Intelligent Energy, 2024, 46(3): 12-19. doi: 10.3969/j.issn.2097-0706.2024.03.002
[3]	樊颜搏, 熊亚选, 李想, 等. 基于遗传算法的建筑用能多目标优化应用进展[J]. 综合智慧能源, 2024, 46(9): 69-85. doi: 10.3969/j.issn.2097-0706.2024.09.009
	FAN Yanbo, XIONG Yaxuan, LI Xiang, et al. Advancement in multi-objective optimization for building energy use based on genetic algorithms[J]. Integrated Intelligent Energy, 2024, 46(9): 69-85. doi: 10.3969/j.issn.2097-0706.2024.09.009
[4]	JAMSHIDI E J, YUSUP Y, KAYODE J S, et al. Detecting outliers in a univariate time series dataset using unsupervised combined statistical methods: A case study on surface water temperature[J]. Ecological Informatics, 2022, 69: 101672.
[5]	顾国庆, 李晓辉. 基于箱线图异常检测的指数加权平滑预测模型[J]. 计算机与现代化, 2021(1): 28-33.
	GU Guoqing, LI Xiaohui. Exponential weighted smoothing prediction model based on abnormal detection of box-plot[J]. Computer and Modernization, 2021(1): 28-33.
[6]	ZHANG J F, ZHANG H, DING S, et al. Power consumption predicting and anomaly detection based on transformer and K-means[J]. Frontiers in Energy Research, 2021, 9: 779587.
[7]	JIN F, WU H, LIU Y, et al. Varying-scale HCA-DBSCAN-based anomaly detection method for multi-dimensional energy data in steel industry[J]. Information Sciences, 2023, 647: 119479.
[8]	LIU X, DING Y, TANG H, et al. A data mining-based framework for the identification of daily electricity usage patterns and anomaly detection in building electricity consumption data[J]. Energy and Buildings, 2021, 231: 110601.
[9]	王锐. 基于改进LOF的高维数据异常检测方法[J]. 电信工程技术与标准化, 2023, 36(3):41-45,62.
	WANG Rui. Anomaly detection for high-dimensional data based on improved LOF[J]. Telecom Engineering Technics and Standardization, 2003, 36(3):41-45,62.
[10]	LESOUPLE J, BAUDOIN C, SPIGAI M, et al. Generalized isolation forest for anomaly detection[J]. Pattern Recognition Letters, 2021, 149: 109-119.
[11]	WANG X L, AHN S H. Real-time prediction and anomaly detection of electrical load in a residential community[J]. Applied Energy, 2020, 259: 114145.
[12]	LI K K, TANG L P, YANG X M. Alleviating limit cycling in training GANs with an optimization technique[J]. Science China Mathematics, 2024, 67(6): 1287-1316.
[13]	陆旦宏, 范文尧, 杨婷, 等. 基于生成对抗Transformer的电力负荷数据异常检测[J]. 电力工程技术, 2024, 43(1): 157-164.
	LU Danhong, FAN Wenyao, YANG Ting, et al. Anomaly detection of power load data based on Transformer and generative adversarial networks[J]. Electric Power Engineering Technology, 2024, 43(1): 157-164.
[14]	曾静, 娄冰, 吕娜, 等. 基于多特征和LOF的用户负荷突变检测[J]. 浙江电力, 2023, 42(2):90-97.
	ZENG Jing, LOU Bing, LYU Na, et al. Abrupt user load change detection based on multiple features and LOF algorithm[J]. Zhejiang Electric Power, 2023, 42(2):90-97.
[15]	HIMEUR Y, ALSALEMI A, BENSAALI F, et al. Smart power consumption abnormality detection in buildings using micromoments and improved K-nearest neighbors[J]. International Journal of Intelligent Systems, 2021, 36(6): 2865-2894.
[16]	XU S Y, LIU H Y, DUAN L T, et al. An improved LOF outlier detection algorithm[C]//Proceedings of 2021 IEEE International Conference on Artificial Intelligence and Computer Applications(ICAICA), IEEE, 2021: 113-117.
[17]	SOLÍS-VILLARREAL J A, SOTO-MENDOZA V, NAVARRO-ACOSTA J A, et al. Energy consumption outlier detection with AI models in modern cities: A case study from north-eastern Mexico[J]. Algorithms, 2024, 17(8): 322.
[18]	李沛智, 刘立群, 薄轶帅. 基于密度子空间孤立森林的异常用电检测[J]. 科学技术与工程, 2024, 24(10): 4115-4123.
	LI Peizhi, LIU Liqun, BO Yishuai. Anomalous electricity usage detection based on density subspace isolated forest[J]. Science Technology and Engineering, 2024, 24(10): 4115-4123.
[19]	肖碧涛, 刘宇, 赖晓路. 基于动态潜变量回归的风电机组叶片结冰监测研究[J]. 综合智慧能源, 2024, 46(12): 29-35. doi: 10.3969/j.issn.2097-0706.2024.12.004
	XIAO Bitao, LIU Yu, LAI Xiaolu. Wind turbine blades icing monitoring based on dynamic latent variable regression[J]. Integrated Intelligent Energy, 2024, 46(12): 29-35. doi: 10.3969/j.issn.2097-0706.2024.12.004
[20]	赵荣珍, 赵楠. 结合扰动集成RBF的故障识别方法[J]. 兰州理工大学学报, 2022, 48(6): 40-45.
	ZHAO Rongzhen, ZHAO Nan. Fault identification method combined with disturbance ensemble RBF[J]. Journal of Lanzhou University of Technology, 48(6): 40-45.
[21]	BA P F, ZHU S L, CHAI H Y, et al. Structural monitoring data repair based on a long short-term memory neural network[J]. Scientific Reports, 2024, 14: 9974. doi: 10.1038/s41598-024-60196-2 pmid: 38693161
[22]	林范龙, 林昀, 黄庆仕, 等. 智能电网中的异常数据修复模型研究[J]. 微型电脑应用, 2024, 40(9): 134-138.
	LIN Fanlong, LIN Yun, HUANG Qingshi, et al. Research on abnormal data repair model in smart grid[J]. Microcomputer Applications, 2024, 40(9): 134-138.
[23]	ZHANG S C. Challenges in KNN classification[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(10): 4663-4675.
[24]	HONG G, CHOI G S, EUM J Y, et al. The hourly energy consumption prediction by KNN for buildings in community buildings[J]. Buildings, 2022, 12(10): 1636.
[25]	YAN H Y, MA L D, ZHAO T Y, et al. Research on repair method of abnormal energy consumption data of lighting and plug based on similar features[J]. Energy and Buildings, 2022, 268: 112155.
[26]	刘立巍, 周建新, 刘培栋, 等. 基于KNN-LSTM的区域热负荷短期预测及在机组热电可行域的应用研究[J]. 热能动力工程, 2023, 38(3):91-97.
	LIU Liwei, ZHOU Jianxin, LIU Peidong, et al. Short-term regional heat load prediction using KNN-LSTM and its application in CHP feasible operation region[J]. Journal of Engineering for Thermal Energy and Power, 2023, 38(3):91-97.
[27]	许艺颖. 建筑能耗监测平台异常数据的辨识与修复[D]. 大连: 大连理工大学, 2019.
	XU Yiying. Identification and repair of abnormal data in energy consumption monitoring platform[D]. Dalian: Dalian University of Technology, 2019.

异常情况	具体表现形式
无数据	在某个时间段内没有任何能耗数据上传
“假”数据	数据维持长期固定不变，不符合客观规律
数据过小	数据反向或数值过小，不符合实际
数据过大	数据数值过大，不符合实际
与相似建筑数据偏差过大	与功能、规模、使用方式相似的其他建筑相比，能耗数据存在不合理差异
与历史数据偏差过大	与历史能耗数据相比，某一时段的能耗数据出现明显异常波动
不符合周期性规律	能耗数据通常具有一定的周期性，若能耗数据在周期性规律上出现明显违背的情况，则可能存在异常

评价标准	8月工作日	8月周末	10月工作日	10月周末
人工验证准确率/％	98.78	97.66	99.48	99.31
A_SS	2.535	2.805	1.938	2.067

模型	办公公共建筑		商业公共建筑
模型	RMSE/kW	MAE	RMSE/kW	MAE
XGBoost	97.277 9	66.267 5	97.346 7	67.144 7
LSTM	85.846 3	66.032 4	86.228 6	67.068 3
BP	177.869 3	126.422 0	175.533 1	128.534 1
SVM	126.991 6	95.372 1	129.211 0	95.538 3

模型	办公公共建筑		商业公共建筑
模型	RMSE/kW	MAE	RMSE/kW	MAE
XGBoost	92.398 2	61.398 2	92.453 7	65.506 9
LSTM	70.556 2	57.624 9	70.853 4	58.164 5
BP	152.249 9	109.552 4	152.357 6	112.438 4
SVM	114.463 2	89.463 2	116.452 2	90.233 8