• HOME
  • About Journal
    • Historical evolution
    • Journal Honors
  • Editorial Board
    • Members of Committee
    • Director of the Committee
    • President and Editor in chief
  • Submission Guide
    • Instructions for Authors
    • Manuscript Processing Flow
    • Model Text
    • Procedures for Submission
  • Academic Influence
  • Open Access
  • Ethics&Policies
    • Publication Ethics Statement
    • Peer Review Process
    • Academic Misconduct Identification and Treatment
    • Advertising and Marketing
    • Correction and Retraction
    • Conflict of Interest
    • Authorship & Copyright
  • Contact Us
  • Chinese
Site search        
文章摘要
基于Spark框架的电力大数据清洗模型
A Data Cleaning Model for Electric Power Big Data Based on Spark Framework
Received:September 06, 2016  Revised:September 06, 2016
DOI:
中文关键词: 电力大数据  数据清洗  异常识别  异常修正  Spark框架
英文关键词: Electric  power big  data, Data  cleaning, Anomaly  identification, Anomaly  modification
基金项目:国家自然科学基金项目( 重点项目)
Author NameAffiliationE-mail
wangchong* Informationg&Telecommunication Branch Company,State Grid East Inner Mongolia Electric Power CO.LTD wangchongky@163.com 
zhangwenlong Changchun Power Supply Company,State Grid Jilin Electric Electric Power CO.LTD 396010868@qq.com 
wangyongwen The Hyflux of Tianjin Electric Co., Ltd 974441891@qq.com 
Hits: 3213
Download times: 1520
中文摘要:
      对电力大数据清洗可提高电力大数据质量的正确性、完整性、一致性、可靠性。针对电力大数据清洗过程中的提取统一异常检测模式困难、异常数据修正连续性及准确性低下等问题,提出了一种基于Spark框架的电力大数据清洗模型。首先基于改进CURE聚类算法获取正常簇;其次,实现了正常簇的边界样本获取方法,并设计了基于边界样本的异常识别算法;最后通过指数加权移动平均数实现了异常数据修正。通过对某风电场风力发电监测数据进行了数据清洗实验分析,验证了清洗模型的高效性、准确性。
英文摘要:
      . The data cleaning of electrical power big data can improve the correctness, the completeness, the consistency and the reliability of the data. Aiming at the difficulties of the extracting of the unified anomaly detection pattern and the low accuracy and continuity of the anomaly data correction in the process of the electrical power big data cleaning, the data cleaning model of the electrical power big data based on Spark is proposed. Firstly, the normal clusters and the corresponding boundary samples are obtained by the improved CURE clustering algorithm. Then, the anomaly data identification algorithm based on boundary samples is designed. Finally, the anomaly data modification is realized by using exponential weighting moving mean value. The high efficiency and accuracy is proved by the experiment of the data cleaning of the wind power generation monitoring data from the wind power station.
View Full Text   View/Add Comment  Download reader
Close
  • Home
  • About Journal
    • Historical evolution
    • Journal Honors
  • Editorial Board
    • Members of Committee
    • Director of the Committee
    • President and Editor in chief
  • Submission Guide
    • Instructions for Authors
    • Manuscript Processing Flow
    • Model Text
    • Procedures for Submission
  • Academic Influence
  • Open Access
  • Ethics&Policies
    • Publication Ethics Statement
    • Peer Review Process
    • Academic Misconduct Identification and Treatment
    • Advertising and Marketing
    • Correction and Retraction
    • Conflict of Interest
    • Authorship & Copyright
  • Contact Us
  • 中文页面
Address: No.2000, Chuangxin Road, Songbei District, Harbin, China    Zip code: 150028
E-mail: dcyb@vip.163.com    Telephone: 0451-86611021
© 2012 Electrical Measurement & Instrumentation
黑ICP备11006624号-1
Support:Beijing Qinyun Technology Development Co., Ltd