Volume 11 Number 5 (May 2016)
Home > Archive > 2016 > Volume 11 Number 5 (May 2016) >
JSW 2016 Vol.11(5): 440-454 ISSN: 1796-217X
doi: 10.17706/jsw.11.5.440-454

Impact of Variances of Random Weights and Biases on Extreme Learning Machine

Xiao Tao1, Xu Zhou2, Yu Lin He3*, Rana Aamir Raza Ashfaq4

1College of Science, Agricultural University of Hebei, Baoding 071001, China.
2Department of Basic Courses, Agricultural University of Hebei, Huanghua 061100, China.
3College of Computer Science & Software Engineering, Shenzhen University, Shenzhen 518060, China.
4Department of Computer Science, Bahauddin Zakariya University, Multan, Pakistan.

Abstract—Although the uniform convergence of extreme learning machine (ELM) has been proved for any continues probability distribution, the variances of random numbers initializing input-layer weights and hidden-layer biases indeed have the obvious impact on generalization performance of ELM. In this paper, we validate this effect by testing the classification accuracies of ELMs initialized by the random numbers with different variances. We select three commonly-used probability distributions (i.e., Uniform, Gamma and Normal) and 30 UCI data sets to conduct our comparative study. The experimental results present some important and valuable observations and instructions: (1) Uniform and Gamma distributions with the smaller variances usually make ELMs get the higher training and testing accuracies; (2) In comparison with Normal distribution, the variances of Uniform and Gamma distributions have the significant impact on classification performance of ELMs; (3) Uniform and Gamma distributions with the larger variances could seriously degrade the classification capability of ELMs; (4) ELMs initialized by Uniform and Gamma distributions with the larger variances generally needs the more hidden-layer nodes to achieve the equivalent classification accuracies with ones having the smaller variances; and (5) Normal distribution are more easily lead to the over-fitting of ELMs.

Index Terms—Extreme learning machine, random initialization, probability distribution, variance.

[PDF]

Cite: Xiao Tao, Xu Zhou, Yu Lin He, Rana Aamir Raza Ashfaq4, "Impact of Variances of Random Weights and Biases on Extreme Learning Machine," Journal of Software vol. 11, no. 5, pp. 440-454, 2016.

General Information

ISSN: 1796-217X (Online)
Frequency:  Quarterly
Editor-in-Chief: Prof. Antanas Verikas
Executive Editor: Ms. Yoyo Y. Zhou
Abstracting/ Indexing: DBLP, EBSCO, CNKIGoogle Scholar, ProQuest, INSPEC(IET), ULRICH's Periodicals Directory, WorldCat, etc
E-mail: jsweditorialoffice@gmail.com
  • Apr 26, 2021 News!

    Vol 14, No 4- Vol 14, No 12 has been indexed by IET-(Inspec)     [Click]

  • Nov 18, 2021 News!

    Papers published in JSW Vol 16, No 1- Vol 16, No 6 have been indexed by DBLP   [Click]

  • Dec 24, 2021 News!

     Vol 15, No 1- Vol 15, No 6 has been indexed by IET-(Inspec)   [Click]

  • Jan 04, 2024 News!

    JSW will adopt Article-by-Article Work Flow

  • Dec 06, 2019 News!

    Vol 14, No 1- Vol 14, No 4 has been indexed by EI (Inspec)   [Click]