Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks

Chloe M. Barnes; Aniko Ekart; Kai Olav Ellefsen; Kyrre Glette; Peter R. Lewis; Jim Torresen

doi:10.1109/ACSOS49614.2020.00034

Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks

Chloe M. Barnes^*, Aniko Ekart, Kai Olav Ellefsen, Kyrre Glette, Peter R. Lewis, Jim Torresen

^*Corresponding author for this work

Universitetet i Oslo

Research output: Chapter in Book/Published conference output › Conference publication

3 Citations (Scopus)

Abstract

Neural networks have been widely used in agent learning architectures; however, learning multiple context dependent tasks simultaneously or sequentially is problematic when using them. Behavioural plasticity enables humans and animals alike to respond to changes in context and environmental stimuli, without degrading learnt knowledge; this can be achieved by regulating behaviour with neuromodulation - a biological process found in the brain. We demonstrate that modulating activity-propagating signals when evolving neural networks enables agents to learn context-dependent and multi-stage tasks more easily. Further, we show that this benefit is preserved when agents occupy an environment shared with other neuromodulated agents. Additionally we show that neuromodulation helps agents that have evolved alone to adapt to changes in environmental stimuli when they continue to evolve in a shared environment.

Original language	English
Title of host publication	Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020
Editors	Esam El-Araby, Sven Tomforde, Timothy Wood, Pradeep Kumar, Claudia Raibulet, Ioan Petri, Gabriele Valentini, Phyllis Nelson, Barry Porter
Publisher	IEEE
Pages	129-138
Number of pages	10
ISBN (Electronic)	9781728172774
ISBN (Print)	9781728172774
DOIs	https://doi.org/10.1109/ACSOS49614.2020.00034
Publication status	Published - 15 Sept 2020
Event	1st IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020 - Virtual, Washington, United States Duration: 17 Aug 2020 → 21 Aug 2020

Publication series

Name	Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020

Conference

Conference	1st IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020
Country/Territory	United States
City	Virtual, Washington
Period	17/08/20 → 21/08/20

Funding

This work was partially supported by the Research Council of Norway through its Centres of Excellence scheme, project number 262762. The authors would like to thank Aston University and the University of Oslo for supporting the research visits during this collaboration.

Access to Document

10.1109/ACSOS49614.2020.00034

Cite this

Barnes, C. M., Ekart, A., Ellefsen, K. O., Glette, K., Lewis, P. R., & Torresen, J. (2020). Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks. In E. El-Araby, S. Tomforde, T. Wood, P. Kumar, C. Raibulet, I. Petri, G. Valentini, P. Nelson, & B. Porter (Eds.), Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020 (pp. 129-138). Article 9196458 (Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020). IEEE. https://doi.org/10.1109/ACSOS49614.2020.00034

Barnes, Chloe M. ; Ekart, Aniko ; Ellefsen, Kai Olav et al. / Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks. Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020. editor / Esam El-Araby ; Sven Tomforde ; Timothy Wood ; Pradeep Kumar ; Claudia Raibulet ; Ioan Petri ; Gabriele Valentini ; Phyllis Nelson ; Barry Porter. IEEE, 2020. pp. 129-138 (Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020).

@inproceedings{039b53c4d6354753abc893102af1e1ff,

title = "Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks",

abstract = "Neural networks have been widely used in agent learning architectures; however, learning multiple context dependent tasks simultaneously or sequentially is problematic when using them. Behavioural plasticity enables humans and animals alike to respond to changes in context and environmental stimuli, without degrading learnt knowledge; this can be achieved by regulating behaviour with neuromodulation - a biological process found in the brain. We demonstrate that modulating activity-propagating signals when evolving neural networks enables agents to learn context-dependent and multi-stage tasks more easily. Further, we show that this benefit is preserved when agents occupy an environment shared with other neuromodulated agents. Additionally we show that neuromodulation helps agents that have evolved alone to adapt to changes in environmental stimuli when they continue to evolve in a shared environment. ",

author = "Barnes, {Chloe M.} and Aniko Ekart and Ellefsen, {Kai Olav} and Kyrre Glette and Lewis, {Peter R.} and Jim Torresen",

year = "2020",

month = sep,

day = "15",

doi = "10.1109/ACSOS49614.2020.00034",

language = "English",

isbn = "9781728172774",

series = "Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020",

publisher = "IEEE",

pages = "129--138",

editor = "Esam El-Araby and Sven Tomforde and Timothy Wood and Pradeep Kumar and Claudia Raibulet and Ioan Petri and Gabriele Valentini and Phyllis Nelson and Barry Porter",

booktitle = "Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020",

address = "United States",

note = "1st IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020 ; Conference date: 17-08-2020 Through 21-08-2020",

}

Barnes, CM , Ekart, A, Ellefsen, KO, Glette, K, Lewis, PR & Torresen, J 2020, Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks. in E El-Araby, S Tomforde, T Wood, P Kumar, C Raibulet, I Petri, G Valentini, P Nelson & B Porter (eds), Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020., 9196458, Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020, IEEE, pp. 129-138, 1st IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020, Virtual, Washington, United States, 17/08/20. https://doi.org/10.1109/ACSOS49614.2020.00034

Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks. / Barnes, Chloe M.; Ekart, Aniko; Ellefsen, Kai Olav et al.
Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020. ed. / Esam El-Araby; Sven Tomforde; Timothy Wood; Pradeep Kumar; Claudia Raibulet; Ioan Petri; Gabriele Valentini; Phyllis Nelson; Barry Porter. IEEE, 2020. p. 129-138 9196458 (Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020).

Research output: Chapter in Book/Published conference output › Conference publication

TY - GEN

T1 - Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks

AU - Barnes, Chloe M.

AU - Ekart, Aniko

AU - Ellefsen, Kai Olav

AU - Glette, Kyrre

AU - Lewis, Peter R.

AU - Torresen, Jim

PY - 2020/9/15

Y1 - 2020/9/15

N2 - Neural networks have been widely used in agent learning architectures; however, learning multiple context dependent tasks simultaneously or sequentially is problematic when using them. Behavioural plasticity enables humans and animals alike to respond to changes in context and environmental stimuli, without degrading learnt knowledge; this can be achieved by regulating behaviour with neuromodulation - a biological process found in the brain. We demonstrate that modulating activity-propagating signals when evolving neural networks enables agents to learn context-dependent and multi-stage tasks more easily. Further, we show that this benefit is preserved when agents occupy an environment shared with other neuromodulated agents. Additionally we show that neuromodulation helps agents that have evolved alone to adapt to changes in environmental stimuli when they continue to evolve in a shared environment.

AB - Neural networks have been widely used in agent learning architectures; however, learning multiple context dependent tasks simultaneously or sequentially is problematic when using them. Behavioural plasticity enables humans and animals alike to respond to changes in context and environmental stimuli, without degrading learnt knowledge; this can be achieved by regulating behaviour with neuromodulation - a biological process found in the brain. We demonstrate that modulating activity-propagating signals when evolving neural networks enables agents to learn context-dependent and multi-stage tasks more easily. Further, we show that this benefit is preserved when agents occupy an environment shared with other neuromodulated agents. Additionally we show that neuromodulation helps agents that have evolved alone to adapt to changes in environmental stimuli when they continue to evolve in a shared environment.

UR - http://www.scopus.com/inward/record.url?scp=85092705281&partnerID=8YFLogxK

UR - https://ieeexplore.ieee.org/document/9196458

U2 - 10.1109/ACSOS49614.2020.00034

DO - 10.1109/ACSOS49614.2020.00034

M3 - Conference publication

AN - SCOPUS:85092705281

SN - 9781728172774

T3 - Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020

SP - 129

EP - 138

BT - Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020

A2 - El-Araby, Esam

A2 - Tomforde, Sven

A2 - Wood, Timothy

A2 - Kumar, Pradeep

A2 - Raibulet, Claudia

A2 - Petri, Ioan

A2 - Valentini, Gabriele

A2 - Nelson, Phyllis

A2 - Porter, Barry

PB - IEEE

T2 - 1st IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020

Y2 - 17 August 2020 through 21 August 2020

ER -

Barnes CM , Ekart A, Ellefsen KO, Glette K, Lewis PR, Torresen J. Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks. In El-Araby E, Tomforde S, Wood T, Kumar P, Raibulet C, Petri I, Valentini G, Nelson P, Porter B, editors, Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020. IEEE. 2020. p. 129-138. 9196458. (Proceedings - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020). doi: 10.1109/ACSOS49614.2020.00034

Coevolutionary learning of neuromodulated controllers for multi-stage and gamified tasks

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Cite this