Volume 11 Number 5 (May 2016)
Home > Archive > 2016 > Volume 11 Number 5 (May 2016) >
JSW 2016 Vol.11(5): 440-454 ISSN: 1796-217X
doi: 10.17706/jsw.11.5.440-454

Impact of Variances of Random Weights and Biases on Extreme Learning Machine

Xiao Tao1, Xu Zhou2, Yu Lin He3*, Rana Aamir Raza Ashfaq4
1College of Science, Agricultural University of Hebei, Baoding 071001, China.
2Department of Basic Courses, Agricultural University of Hebei, Huanghua 061100, China.
3College of Computer Science & Software Engineering, Shenzhen University, Shenzhen 518060, China.
4Department of Computer Science, Bahauddin Zakariya University, Multan, Pakistan.

Abstract—Although the uniform convergence of extreme learning machine (ELM) has been proved for any continues probability distribution, the variances of random numbers initializing input-layer weights and hidden-layer biases indeed have the obvious impact on generalization performance of ELM. In this paper, we validate this effect by testing the classification accuracies of ELMs initialized by the random numbers with different variances. We select three commonly-used probability distributions (i.e., Uniform, Gamma and Normal) and 30 UCI data sets to conduct our comparative study. The experimental results present some important and valuable observations and instructions: (1) Uniform and Gamma distributions with the smaller variances usually make ELMs get the higher training and testing accuracies; (2) In comparison with Normal distribution, the variances of Uniform and Gamma distributions have the significant impact on classification performance of ELMs; (3) Uniform and Gamma distributions with the larger variances could seriously degrade the classification capability of ELMs; (4) ELMs initialized by Uniform and Gamma distributions with the larger variances generally needs the more hidden-layer nodes to achieve the equivalent classification accuracies with ones having the smaller variances; and (5) Normal distribution are more easily lead to the over-fitting of ELMs.

Index Terms—Extreme learning machine, random initialization, probability distribution, variance.


Cite: Xiao Tao, Xu Zhou, Yu Lin He, Rana Aamir Raza Ashfaq4, "Impact of Variances of Random Weights and Biases on Extreme Learning Machine," Journal of Software vol. 11, no. 5, pp. 440-454, 2016.

General Information

ISSN: 1796-217X (Online)
Frequency:  Bimonthly (Since 2020)
Editor-in-Chief: Prof. Antanas Verikas
Executive Editor: Ms. Yoyo Y. Zhou
Abstracting/ Indexing: DBLP, EBSCO, Google Scholar, ProQuest, INSPEC(IET), ULRICH's Periodicals Directory, WorldCat, etc
E-mail: jsw@iap.org
  • Apr 26, 2021 News!

    Vol 14, No 4- Vol 14, No 12 has been indexed by IET-(Inspec)     [Click]

  • Jun 22, 2020 News!

    Papers published in JSW Vol 14, No 1- Vol 15 No 4 have been indexed by DBLP     [Click]

  • Sep 13, 2021 News!

    The papers published in Vol 16, No 6 have all received dois from Crossref    [Click]

  • Jan 28, 2021 News!

    [CFP] 2021 the annual meeting of JSW Editorial Board, ICCSM 2021, will be held in Rome, Italy, July 21-23, 2021   [Click]

  • Sep 13, 2021 News!

    Vol 16, No 6 has been published with online version     [Click]