郑秀杰,董彬彬,易建波.电网量测数据海量终端的前置中间件技术研究[J].电测与仪表,2020,57(1):70-75. Zhengxiujie,Dongbinbin,Yijianbo.Research on Pre-middleware Technology for Massive Terminal of Power Grid Measurement Data[J].Electrical Measurement & Instrumentation,2020,57(1):70-75.
电网量测数据海量终端的前置中间件技术研究
Research on Pre-middleware Technology for Massive Terminal of Power Grid Measurement Data
Aiming at the massive, multi-source and high-speed data processing problems of WAMS, SCADA, AMI and other measurement systems currently facing monitoring and analysis of smart grid, this paper proposes a data pre-processing middleware technology for mass terminals, which focuses on the efficient mining of target information and processor load balancing in massive data. In the pre-data processing middleware architecture, there designs a parallel mining algorithm based on sampling target information data, and the load is balanced by the idea of parallel computing based on Map-Reduce and the idea of rotation algorithm. With aggregating data inline relationships by sampling mining, this paper designs a single-machine multi-core parallel data mining strategy. Finally, through the comparative test of massive PMU data in wide-area power grid, the results show that the middleware technology can effectively improve the mining speed and multi-processor load balance, and greatly reduce the memory burden in massive data mining.
