田英杰,洪子靖,周李.基于函数型数据分析的工商业居民用户电力数据清洗算法[J].电测与仪表,2021,58(1):11-19. Tian Yingjie,Hong Zijing,Zhou Li.Data cleaning algorithm for industrial and commercial residential users based on functional data analysis[J].Electrical Measurement & Instrumentation,2021,58(1):11-19.
基于函数型数据分析的工商业居民用户电力数据清洗算法
Data cleaning algorithm for industrial and commercial residential users based on functional data analysis
用户电力数据的爆炸式增长给原始数据修正带来许多难点。文章提出用函数型数据分析(Functional Data Analysis, FDA)对错误和缺失数据进行修正与补全。通过函数估计方法，将原有观测个体的离散数据映射到一个新的函数空间，将数据中缺失的成分利用相似用户曲线特征进行修复，并搭建了针对电力大数据的数据清洗整体框架。在真实数据集上的测试结果表明，该算法能够准确地提取用户的用电特征曲线，并对错误数据和缺失数据进行准确地修复。
英文摘要:
The explosive growth of user power data has brought many difficulties to the repairing of raw data. Functional data analysis (FDA) is proposed to complete the missing data and correct the error data in this paper. The observed discrete raw data is mapped to a new function space through the function estimation method. The missing components in the data are repaired by using similar user curve features. This paper also builds the overall framework of data cleaning for power big data. The test results on the real data set show that the proposed algorithm can accurately extract the power consumption characteristic curve of users, accurately repair the error data and complete the missing data.
