Volume 10 Number 3 (Mar. 2015)
Home > Archive > 2015 > Volume 10 Number 3 (Mar. 2015) >
JSW 2015 Vol.10(3): 239-249 ISSN: 1796-217X
doi: 10.17706/jsw.10.3.239-249

Accelerated -Greedy Multi Armed Bandit Algorithm for Online Sequential-Selections Applications

Khosrow Amirizadeh*, Rajeswari Mandava

Computer Vision Lab., School of Computer Sciences, Universiti Sains Malaysia (USM), 11800 Penang, Malaysia

Abstract—Current algorithms for solving multi-armed bandit (MAB) problem in stationary observations often perform well. Although this performance may be acceptable with accurate parameter settings, most of them degrade under non stationary observations. We setup an incremental ε-greedy model with stochastic mean equation as its action-value function which is more applicable to real-world problems. Unlike the iterative algorithms suffering from step size dependency, we propose an adaptive step-size model (ASM) to introduce adaptive MAB algorithm. The proposed model employs ε-greedy approach as action selection policy. In addition, a dynamic exploration parameter ε is introduced to be ineffective by increasing decision maker’s intelligence. The proposed model is empirically evaluated and compared with existing algorithms including the standard ε-greedy, Softmax, ε-decreasing and UCB-Tuned models under stationary as well as non stationary situations. ASM not only addresses concerns in parameter dependency problem but also performs either comparable or better than mentioned algorithms. Applying these enhancements to the standard ε-greedy reduce the learning time which is more attractive to the wide range of on-line sequential selection-based applications such as autonomous agents, adaptive control, industrial robots and forecasting trend problems in management and economics domains.

Index Terms—Enhanced MAB, adaptive incremental learning, MAB empirical evaluations, setting-free step-size model.


Cite: Khosrow Amirizadeh*, Rajeswari Mandava, "Accelerated -Greedy Multi Armed Bandit Algorithm for Online Sequential-Selections Applications," Journal of Software vol. 10, no. 3, pp. 239-249, 2015.

General Information

ISSN: 1796-217X (Online)
Frequency:  Quarterly
Editor-in-Chief: Prof. Antanas Verikas
Executive Editor: Ms. Yoyo Y. Zhou
Abstracting/ Indexing: DBLP, EBSCO, CNKIGoogle Scholar, ProQuest, INSPEC(IET), ULRICH's Periodicals Directory, WorldCat, etc
E-mail: jsweditorialoffice@gmail.com
  • Mar 01, 2024 News!

    Vol 19, No 1 has been published with online version    [Click]

  • Jan 04, 2024 News!

    JSW will adopt Article-by-Article Work Flow

  • Apr 01, 2024 News!

    Vol 14, No 4- Vol 14, No 12 has been indexed by IET-(Inspec)     [Click]

  • Apr 01, 2024 News!

    Papers published in JSW Vol 18, No 1- Vol 18, No 6 have been indexed by DBLP   [Click]

  • Nov 02, 2023 News!

    Vol 18, No 4 has been published with online version   [Click]