Volume 12 Number 12 (Dec. 2017)
Home > Archive > 2017 > Volume 12 Number 12 (Dec. 2017) >
JSW 2017 Vol.12(12): 945-956 ISSN: 1796-217X
doi: 10.17706/jsw.12.12.945-956

An Iterative and Incremental Data Quality Improvement Procedure for Reducing the Risk of Big Data Project

Sen-Tarng Lai*
Department of Information Technology and Management, Shih Chien University, No.70, Dazhi St., Zhongshan Dist., Taipei City 104, Taiwan (R.O.C.)

Abstract—Big data applications can enhance the market competitive advantages of enterprises and organizations and can improve people's quality of life. However, by the impact of many factors, failure rate of big data project is higher than the IT project. In order to reduce the risk of failure, big data projects must overcome a serial of challenges. Ambiguous requirements, poor data quality, and lacking changeability and extensity will directly affect the results of big data analytics. And even cause the wrong decision, inaccurate prediction and improper planning to make the big data projects with potential high risk. For this, this paper migrates iterative and incremental development (IID) features to the data preprocessing, and draws up the iterative and incremental data quality improvement (IIDQI) procedure. IIDQI procedure applies data preprocessing task frame to repeatedly detect and identify the defects of data quality, and incrementally strengthen big data quality and control the factors of failure risk. Iterative inspection activities can effectively enhance data quality, intercommunication efficiency, and precision requirement and objective to reduce the risk of big data project failure.

Index Terms—Big data, data preprocessing, failure risk, IID, quality improvement.


Cite: Sen-Tarng Lai, "An Iterative and Incremental Data Quality Improvement Procedure for Reducing the Risk of Big Data Project," Journal of Software vol. 12, no. 12, pp. 945-956, 2017.

General Information

ISSN: 1796-217X
Frequency: Monthly
Editor-in-Chief: Prof. Antanas Verikas
Executive Editor: Ms. Yoyo Y. Zhou
Abstracting/ Indexing: DBLP, EBSCO, ProQuest, INSPEC, ULRICH's Periodicals Directory, WorldCat, CNKI,etc
E-mail: jsw@iap.org
  • Nov 29, 2018 News!

    Papers published in JSW Vol. 13, No. 1- Vol. 13 No. 10 have been indexed by DBLP.    [Click]

  • Aug 24, 2018 News!

    Vol.12, No.8- Vol.13, No.5 has been indexed by EI (Inspec).   [Click]

  • Aug 01, 2018 News!

    [CFP] 2019 the annual meeting of JSW Editorial Board, ICCSM 2019, will be held in Barcelona, Spain, July 14-16, 2019.   [Click]

  • Nov 08, 2018 News!

    The papers published in Vol.13, No. 10 have all received dois from Crossref.

  • Nov 29, 2018 News!

    Vol 13, No. 12 has been published with online version 4 original aritcles from 3 countries are published in this issue.     [Click]