• HOME
  • About Journal
    • Historical evolution
    • Journal Honors
  • Editorial Board
    • Members of Committee
    • Director of the Committee
    • President and Editor in chief
  • Submission Guide
    • Instructions for Authors
    • Manuscript Processing Flow
    • Model Text
    • Procedures for Submission
  • Academic Influence
  • Open Access
  • Ethics&Policies
    • Publication Ethics Statement
    • Peer Review Process
    • Academic Misconduct Identification and Treatment
    • Advertising and Marketing
    • Correction and Retraction
    • Conflict of Interest
    • Authorship & Copyright
  • Contact Us
  • Chinese
Site search        
文章摘要
基于Hadoop的风力发电监测大数据存储优化及并行查询方法
Storage optimization and parallel query method for big data of wind power monitoring based on Hadoop
Received:June 13, 2017  Revised:June 14, 2017
DOI:
中文关键词: 大数据  风力发电监测  Hadoop  哈希分桶算法
英文关键词: big data, wind power monitoring, Hadoop, hash bucket algorithm
基金项目:国家高技术研究发展计划(863计划)(2015AA050203);国家电网公司科技项目(520900150037)
Author NameAffiliationE-mail
Wang Lintong* Department of Electrical Engineering,Shanghai Jiaotong University 18817555814@163.com 
Zhao Teng Department of Electrical Engineering,Shanghai Jiaotong University zhaoteng@sjtu.edu.cn 
Zhang Yan Department of Electrical Engineering,Shanghai Jiaotong University zhang_yan@sjtu.edu.cn 
Su Yun Electric Power Research Institute,SMEPC oppenvi@163.com 
Hits: 1938
Download times: 740
中文摘要:
      随着风力发电的广泛发展以及智能化监测技术的推广应用,风力发电监测数据呈现出体量大、类型多、增长快的大数据特征。针对风力发电监测大数据高效存储和快速查询两方面核心问题,本文基于Hadoop平台进行大数据存储优化方法研究,提出考虑风力发电监测数据关联性的哈希分桶存储算法,实现了相关联数据的集中存储,从而提升后期数据查询及处理的效率。在数据存储优化的基础上,实现基于MapReduce的多源风力发电监测大数据并行关联查询。通过在Hadoop平台上进行测试表明,经过哈希分桶存储优化后的多源数据并行关联查询相比传统Hadoop方法查询时间显著缩短。
英文摘要:
      With the extensive development of wind power generation and the generalized application of intelligent monitoring technology, wind power monitoring data shows the big data characteristics of large volume, multi types and fast growth. In order to solve the two major problems with big data efficient storage and quick query, in this paper, the optimization method of big data storage is studied based on Hadoop platform. A Hash bucket algorithm considering wind power monitoring data correlation is proposed. The algorithm realizes the centralized storage of related data, so as to enhance the efficiency of data query and processing. On the basis of data storage optimization, the parallel association query for multi-source big data of wind power monitoring based on MapReduce is realized. Tests on a Hadoop platform show that, after optimization of hash bucket storage, the time of the multi-source data parallel association query is significantly shortened than traditional Hadoop method.
View Full Text   View/Add Comment  Download reader
Close
  • Home
  • About Journal
    • Historical evolution
    • Journal Honors
  • Editorial Board
    • Members of Committee
    • Director of the Committee
    • President and Editor in chief
  • Submission Guide
    • Instructions for Authors
    • Manuscript Processing Flow
    • Model Text
    • Procedures for Submission
  • Academic Influence
  • Open Access
  • Ethics&Policies
    • Publication Ethics Statement
    • Peer Review Process
    • Academic Misconduct Identification and Treatment
    • Advertising and Marketing
    • Correction and Retraction
    • Conflict of Interest
    • Authorship & Copyright
  • Contact Us
  • 中文页面
Address: No.2000, Chuangxin Road, Songbei District, Harbin, China    Zip code: 150028
E-mail: dcyb@vip.163.com    Telephone: 0451-86611021
© 2012 Electrical Measurement & Instrumentation
黑ICP备11006624号-1
Support:Beijing Qinyun Technology Development Co., Ltd