An efficient knowledge transfer solution to a novel SMDP formalization of a broker's decision problem

Rodrigue Talla Kuate, Maria Chli, Hai H. Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper introduces a new technique for optimizing the trading strategy of brokers that autonomously trade in re- tail and wholesale markets. Simultaneous optimization of re- tail and wholesale strategies has been considered by existing studies as intractable. Therefore, each of these strategies is optimized separately and their interdependence is generally ignored, with resulting broker agents not aiming for a glob- ally optimal retail and wholesale strategy. In this paper, we propose a novel formalization, based on a semi-Markov deci- sion process (SMDP), which globally and simultaneously op- timizes retail and wholesale strategies. The SMDP is solved using hierarchical reinforcement learning (HRL) in multi- agent environments. To address the curse of dimensionality, which arises when applying SMDP and HRL to complex de- cision problems, we propose an ecient knowledge transfer approach. This enables the reuse of learned trading skills in order to speed up the learning in new markets, at the same time as making the broker transportable across market envi- ronments. The proposed SMDP-broker has been thoroughly evaluated in two well-established multi-agent simulation en- vironments within the Trading Agent Competition (TAC) community. Analysis of controlled experiments shows that this broker can outperform the top TAC-brokers. More- over, our broker is able to perform well in a wide range of environments by re-using knowledge acquired in previously experienced settings.
Original languageEnglish
Title of host publicationAAMAS'15 International Conference on Autonomous Agents and Multi Agent Solutions
PublisherACM
Pages1735-1736
Number of pages2
Volume3
ISBN (Print)978-1-4503-3771-7
Publication statusPublished - 4 May 2015
Event14th International Conference on Autonomous Agents and Multi Agent Systems - Istanbul Congres Center, Istanbul, Turkey
Duration: 4 May 20158 May 2015

Conference

Conference14th International Conference on Autonomous Agents and Multi Agent Systems
Abbreviated titleAAMAS 2015
CountryTurkey
CityIstanbul
Period4/05/158/05/15

    Fingerprint

Keywords

  • artificial intelligence
  • learning
  • knowledge transfer
  • reinforcement learning
  • MDP
  • SMDP
  • broker agent

Cite this

Talla Kuate, R., Chli, M., & Wang, H. H. (2015). An efficient knowledge transfer solution to a novel SMDP formalization of a broker's decision problem. In AAMAS'15 International Conference on Autonomous Agents and Multi Agent Solutions (Vol. 3, pp. 1735-1736). [paper 278] ACM.