Volume 9 Number 10 (Oct. 2014)
Home > Archive > 2014 > Volume 9 Number 10 (Oct. 2014) >
JSW 2014 Vol.9(10): 2598-2606 ISSN: 1796-217X
doi: 10.4304/jsw.9.10.2598-2606

Distributed Storage and Processing Method for Big Data Sensing Information of Machine Operation Condition

Fan Zhang, Zude Zhou, Wenjun Xu

School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China; Key Lab. of Fiber Optic Sensing Technology and Information Processing, Ministry of Education, Wuhan 430070, China

Abstract—The traditional relational database cannot satisfy the requirements of the high speed and real-time storage and processing for the distributed Big Data sensing information in the Wide Area Network environment. In this context, the No-SQL database HBase is used to store the big data sensing information of machine operation condition collected by Fiber Bragg Grating sensor network. The distributed storage environment and the optimal database table scheme is built. Moreover, the HBase Rowkey is designed in detail to sharpen the retrieval speed and avoid the server hot point accumulation. Meanwhile, the real-time outlier detection method with the working situational constraint is proposed to monitor the machine working condition. It is implemented by the multi-dimensional histogram statistics method in the Map Reduce distributed environment. Hence, the traditional threshold monitoring method is improved and the false alarm problem is eliminated. Through balancing the performance between HBase and the relational database MySQL, the real-time storage rate of the proposed method can satisfy at least 20 machines running concurrency with 4000 Hz Fiber Bragg Grating sampling frequency by HBase. Also, the effectiveness of real-time outlier detection method is proved by the practical operation data processing.

Index Terms—Machine operation condition, Fiber Bragg Grating sensing, Big Data, distributed storage and processing

[PDF]

Cite: Fan Zhang, Zude Zhou, Wenjun Xu, "Distributed Storage and Processing Method for Big Data Sensing Information of Machine Operation Condition," Journal of Software vol. 9, no. 10, pp. 2598-2606, 2014.

General Information

ISSN: 1796-217X (Online)
Frequency:  Quarterly
Editor-in-Chief: Prof. Antanas Verikas
Executive Editor: Ms. Yoyo Y. Zhou
Abstracting/ Indexing: DBLP, EBSCO, CNKIGoogle Scholar, ProQuest, INSPEC(IET), ULRICH's Periodicals Directory, WorldCat, etc
E-mail: jsweditorialoffice@gmail.com
  • Mar 01, 2024 News!

    Vol 19, No 1 has been published with online version    [Click]

  • Apr 26, 2021 News!

    Vol 14, No 4- Vol 14, No 12 has been indexed by IET-(Inspec)     [Click]

  • Nov 18, 2021 News!

    Papers published in JSW Vol 16, No 1- Vol 16, No 6 have been indexed by DBLP   [Click]

  • Jan 04, 2024 News!

    JSW will adopt Article-by-Article Work Flow

  • Nov 02, 2023 News!

    Vol 18, No 4 has been published with online version   [Click]