面向新型电力系统的电力大数据副本管理算法

丁斌; 袁博; 郑焕坤; 邢志坤; 王帆

丁斌,袁博,郑焕坤,邢志坤,王帆.面向新型电力系统的电力大数据副本管理算法[J].电测与仪表,2022,59(1):10-17.
丁,YUAN Bo,ZHENG Huan-kun,XING Zhi-kun,WANG Fan.Research on Replica Management Strategy for Adaptive Storage of Electric Power Big Data[J].Electrical Measurement & Instrumentation,2022,59(1):10-17.

面向新型电力系统的电力大数据副本管理算法

Research on Replica Management Strategy for Adaptive Storage of Electric Power Big Data

DOI：10.19753/j.issn1001-1390.2022.01.002

中文关键词: 电力大数据，副本管理低时延流量预测

英文关键词:Power big data, Copy management Low latency Traffic forecast

基金项目:国家自然科学基金资助项目(61501185);国家电网面向多维信息系统智能运维的关键技术研究(SGTYHT/19-JS-215),中央高校基本科研业务费专项资金资助项目(2014MS87)

作者中文名	作者英文名	单位
丁斌	丁	国网河北省电力有限公司雄安新区供电公司
袁博	YUAN Bo	国网河北省电力有限公司雄安新区供电公司
郑焕坤	ZHENG Huan-kun	华北电力大学
邢志坤	XING Zhi-kun	国网河北省电力有限公司雄安新区供电公司
王帆	WANG Fan	国网河北省电力有限公司雄安新区供电公司

摘要点击次数: 1549

中文摘要:

随着新型电力系统建设的不断加快,电网信息化规模持续增长,电力信息系统从数百台服务器发展到庞大的上万台虚拟机的数据中心,单靠人工对运维故障进行发现、梳理、统计、管理,不仅耗时耗力,也很难找到告警信息的根源进行故障溯源。如何实现高效的大数据存储,满足低延迟处理应用的需求,是一个非常具有挑战性的问题。本文提出了一种基于随机配置网络(SCN)的自适应副本存储管理系统,该系统考虑了网络流量和数据中心的地理分布。首先,SCN模型作为一种计算量小、预测性能好的快速学习模型,用于估计电力数据网络的流量状态。然后,提出了一系列的数据副本管理算法,以降低有限的带宽和固定的底层基础设施的影响。最后,利用电力行业数据并行网络计算框架实现了该模型。在相应省公司开展试点验证,该方案能够有效地处理电力大数据存储,跨分布式DCs的作业完成时间平均减少12.19%。

英文摘要:

With the continuous acceleration of the construction of the power Internet of Things, the scale of power grid informatization continues to grow, and the power information system has developed from hundreds of servers to a huge data center with tens of thousands of virtual machines. Statistics and management are not only time-consuming and labor-intensive, but it is also difficult to find the root cause of the alarm information to trace the source of the fault. How to achieve efficient big data storage and meet the needs of low-latency processing applications is a very challenging problem. This paper proposes an adaptive power storage replica management system based on random configuration network (SCN), which takes into account the network traffic and the geographical distribution of data centers, and improves the real-time performance of data. First of all, the SCN model is used as a fast learning model with a small amount of calculation and good predictive performance to estimate the flow status of the power data network. Then, a series of data copy management algorithms are proposed to reduce the impact of limited bandwidth and fixed underlying infrastructure. Finally, the model is implemented using Data Parallel Computing Frameworks (DCFs) in the power industry. Pilot verification was carried out in the corresponding provincial company, and the program can effectively handle the large data storage of electric power, and the completion time of operations across distributed DCs was reduced by 12.19% on average.

查看全文查看/发表评论下载PDF阅读器

关闭