Volume 10 Number 3 (Mar. 2015)
Home > Archive > 2015 > Volume 10 Number 3 (Mar. 2015) >
JSW 2015 Vol.10(3): 239-249 ISSN: 1796-217X
doi: 10.17706/jsw.10.3.239-249

Accelerated -Greedy Multi Armed Bandit Algorithm for Online Sequential-Selections Applications

Khosrow Amirizadeh*, Rajeswari Mandava
Computer Vision Lab., School of Computer Sciences, Universiti Sains Malaysia (USM), 11800 Penang, Malaysia

Abstract—Current algorithms for solving multi-armed bandit (MAB) problem in stationary observations often perform well. Although this performance may be acceptable with accurate parameter settings, most of them degrade under non stationary observations. We setup an incremental ε-greedy model with stochastic mean equation as its action-value function which is more applicable to real-world problems. Unlike the iterative algorithms suffering from step size dependency, we propose an adaptive step-size model (ASM) to introduce adaptive MAB algorithm. The proposed model employs ε-greedy approach as action selection policy. In addition, a dynamic exploration parameter ε is introduced to be ineffective by increasing decision maker’s intelligence. The proposed model is empirically evaluated and compared with existing algorithms including the standard ε-greedy, Softmax, ε-decreasing and UCB-Tuned models under stationary as well as non stationary situations. ASM not only addresses concerns in parameter dependency problem but also performs either comparable or better than mentioned algorithms. Applying these enhancements to the standard ε-greedy reduce the learning time which is more attractive to the wide range of on-line sequential selection-based applications such as autonomous agents, adaptive control, industrial robots and forecasting trend problems in management and economics domains.

Index Terms—Enhanced MAB, adaptive incremental learning, MAB empirical evaluations, setting-free step-size model.

[PDF]

Cite: Khosrow Amirizadeh*, Rajeswari Mandava, "Accelerated -Greedy Multi Armed Bandit Algorithm for Online Sequential-Selections Applications," Journal of Software vol. 10, no. 3, pp. 239-249, 2015.

General Information

ISSN: 1796-217X (Online)
Frequency: Monthly
Editor-in-Chief: Prof. Antanas Verikas
Executive Editor: Ms. Yoyo Y. Zhou
Abstracting/ Indexing: DBLP, EBSCO, ProQuest, INSPEC, ULRICH's Periodicals Directory, WorldCat, etc
E-mail: jsw@iap.org
  • Aug 21, 2019 News!

    Papers published in JSW Vol 14, No 1- Vol 14 No 8 have been indexed by DBLP     [Click]

  • Jun 25, 2019 News!

    Vol.13, No.9 has been indexed by EI (Inspec).   [Click]

  • Aug 01, 2018 News!

    [CFP] 2019 the annual meeting of JSW Editorial Board, ICCSM 2019, will be held in Barcelona, Spain, July 14-16, 2019.   [Click]

  • Jul 10, 2019 News!

    Vol 14, No.8 has been published with online version 4 original aritcles from 2 countries are published in this issue.    [Click]

  • Jul 22, 2019 News!

    Welcome Prof Ferhat Khendek from Canada to join the Editorial board of JSW    [Click]