Volume 10 Number 3 (Mar. 2015)
Home > Archive > 2015 > Volume 10 Number 3 (Mar. 2015) >
JSW 2015 Vol.10(3): 239-249 ISSN: 1796-217X
doi: 10.17706/jsw.10.3.239-249

Accelerated -Greedy Multi Armed Bandit Algorithm for Online Sequential-Selections Applications

Khosrow Amirizadeh*, Rajeswari Mandava
Computer Vision Lab., School of Computer Sciences, Universiti Sains Malaysia (USM), 11800 Penang, Malaysia

Abstract—Current algorithms for solving multi-armed bandit (MAB) problem in stationary observations often perform well. Although this performance may be acceptable with accurate parameter settings, most of them degrade under non stationary observations. We setup an incremental ε-greedy model with stochastic mean equation as its action-value function which is more applicable to real-world problems. Unlike the iterative algorithms suffering from step size dependency, we propose an adaptive step-size model (ASM) to introduce adaptive MAB algorithm. The proposed model employs ε-greedy approach as action selection policy. In addition, a dynamic exploration parameter ε is introduced to be ineffective by increasing decision maker’s intelligence. The proposed model is empirically evaluated and compared with existing algorithms including the standard ε-greedy, Softmax, ε-decreasing and UCB-Tuned models under stationary as well as non stationary situations. ASM not only addresses concerns in parameter dependency problem but also performs either comparable or better than mentioned algorithms. Applying these enhancements to the standard ε-greedy reduce the learning time which is more attractive to the wide range of on-line sequential selection-based applications such as autonomous agents, adaptive control, industrial robots and forecasting trend problems in management and economics domains.

Index Terms—Enhanced MAB, adaptive incremental learning, MAB empirical evaluations, setting-free step-size model.


Cite: Khosrow Amirizadeh*, Rajeswari Mandava, "Accelerated -Greedy Multi Armed Bandit Algorithm for Online Sequential-Selections Applications," Journal of Software vol. 10, no. 3, pp. 239-249, 2015.

General Information

ISSN: 1796-217X (Online)
Frequency: Monthly (2006-2019); Bimonthly (Since 2020)
Editor-in-Chief: Prof. Antanas Verikas
Executive Editor: Ms. Yoyo Y. Zhou
Abstracting/ Indexing: DBLP, EBSCO, Google Scholar, ProQuest, INSPEC, ULRICH's Periodicals Directory, WorldCat, etc
E-mail: jsw@iap.org
  • Dec 06, 2019 News!

    Vol 14, No 1- Vol 14, No 4 has been indexed by EI (Inspec)   [Click]

  • Jun 22, 2020 News!

    Papers published in JSW Vol 14, No 1- Vol 15 No 4 have been indexed by DBLP     [Click]

  • Jun 22, 2020 News!

    The papers published in Vol 15, No 5 have all received dois from Crossref    [Click]

  • Aug 01, 2018 News!

    [CFP] 2020 the annual meeting of JSW Editorial Board, ICCSM 2020, will be held in Rome, Italy, July 17-19, 2020   [Click]

  • Jun 22, 2020 News!

    Vol 15, No 5 has been published with online version     [Click]