Volume 7 Number 5 (May. 2012)
Home > Archive > 2012 > Volume 7 Number 5 (May. 2012) >
JSW 2012 Vol.7(5): 943-950 ISSN: 1796-217X
doi: 10.4304/jsw.7.5.943-950

Continuous-time MAXQ Algorithm for Web Service Composition

Hao Tang1, Wenjing Liu2, Wenjuan Cheng2, and Lei Zhou2
1School of Electrical Engineering and Automation, Hefei University of Technology, Tunxi Road No.193, Hefei, Anhui 230009, P.R. China
2School of Computer and Information, Hefei University of Technology, Tunxi Road No.193, Hefei, Anhui 230009, P.R. China

Abstract—Web services composition present a technology to compose complex service applications from individual (atomic) services, that is, through web services composition, distributed applications and enterprise business processes can be integrated by individual service components developed independently. In this paper, we concentrate on the optimization problems of dynamic web service composition, and our goal is to find an optimal composite policy. Different from many traditional composite methods that do not scale to large continuous-time processes, we introduce a hierarchical reinforcement learning technique, i.e., a continuous-time unified MAXQ algorithm, to solve large-scale web service composition problems in the context of continuous-time semi-Markov decision process (SMDP) model under either average- or discounted-cost criteria. The proposed algorithm can avoid the “curse of modeling” and the “curse of dimensionality” existing in the optimization process. Finally, we use a travel reservation as an example to illustrate the high effectiveness of the proposed algorithm, and the simulation results show that, it has better optimization performance and faster learning speed than the flat Q-learning.

Index Terms—web service composition, hierarchical reinforcement learning, semi-Markov decision process (SMDP), MAXQ


Cite: Hao Tang, Wenjing Liu, Wenjuan Cheng, and Lei Zhou "Continuous-time MAXQ Algorithm for Web Service Composition," Journal of Software vol. 7, no. 5, pp. 943-950, 2012.

General Information

ISSN: 1796-217X (Online)
Frequency: Monthly (2006-2019); Bimonthly (Since 2020)
Editor-in-Chief: Prof. Antanas Verikas
Executive Editor: Ms. Yoyo Y. Zhou
Abstracting/ Indexing: DBLP, EBSCO, Google Scholar, ProQuest, INSPEC, ULRICH's Periodicals Directory, WorldCat, etc
E-mail: jsw@iap.org
  • Dec 06, 2019 News!

    Vol 14, No 1- Vol 14, No 4 has been indexed by EI (Inspec)   [Click]

  • Jun 22, 2020 News!

    Papers published in JSW Vol 14, No 1- Vol 15 No 4 have been indexed by DBLP     [Click]

  • Jun 22, 2020 News!

    The papers published in Vol 15, No 5 have all received dois from Crossref    [Click]

  • Aug 01, 2018 News!

    [CFP] 2020 the annual meeting of JSW Editorial Board, ICCSM 2020, will be held in Rome, Italy, July 17-19, 2020   [Click]

  • Jun 22, 2020 News!

    Vol 15, No 5 has been published with online version     [Click]