Towards Model-Driven Self-Explanation for Autonomous Decision-Making Systems

Owen Reynolds

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The ability for systems to make decisions by themselves is increasing with advances in different areas of AI such as machine learning and optimisation techniques for autonomous systems among other. Humans are handing over more decisions to systems that provide no explanations for their judgements unless they are enabled explicitly in their design. Trust based on a program being well written and tested correctly is not appropriate for AI-based autonomous systems. Unlike traditional software, this new software increasingly exhibit emergent behaviours making it unpredictable due to unexpected situations. Self-explanation is sometimes implemented, tracking decisions to give explanations to users. A more consistent, proven approach to self-explanation would be needed for making trustable systems. The paper proposes a research agenda to define an architecture to enable self-explanation for autonomous decision-making systems. The approach will be model-driven to facilitate reuse, the rapid development of tools and suitable abstractions for demonstrating concepts. The architecture will be informed by existing research in provenance ontology and model version research. The evaluation of the architecture is expected to be done using two case studies. The first will implement self-explanation as a primary concern in the building of a system. The second case will attempt to fit self-explanation to an existing system.
Original languageEnglish
Title of host publicationProceedings - 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2019
EditorsLoli Burgueno, Loli Burgueno, Alexander Pretschner, Sebastian Voss, Michel Chaudron, Jorg Kienzle, Markus Volter, Sebastien Gerard, Mansooreh Zahedi, Erwan Bousse, Arend Rensink, Fiona Polack, Gregor Engels, Gerti Kappel
PublisherIEEE
Pages624-628
Number of pages5
ISBN (Electronic)978-1-7281-5125-0
ISBN (Print)978-1-7281-5126-7
DOIs
Publication statusPublished - 21 Nov 2019
Event2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C) - Munich, Germany
Duration: 15 Sep 201920 Sep 2019

Conference

Conference2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)
Period15/09/1920/09/19

Fingerprint

Decision making
Ontology
Learning systems

Keywords

  • Autonomous
  • Decision-making
  • Model-driven
  • Self-explanation

Cite this

Reynolds, O. (2019). Towards Model-Driven Self-Explanation for Autonomous Decision-Making Systems. In L. Burgueno, L. Burgueno, A. Pretschner, S. Voss, M. Chaudron, J. Kienzle, M. Volter, S. Gerard, M. Zahedi, E. Bousse, A. Rensink, F. Polack, G. Engels, ... G. Kappel (Eds.), Proceedings - 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2019 (pp. 624-628). [8904475] IEEE. https://doi.org/10.1109/MODELS-C.2019.00095
Reynolds, Owen. / Towards Model-Driven Self-Explanation for Autonomous Decision-Making Systems. Proceedings - 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2019. editor / Loli Burgueno ; Loli Burgueno ; Alexander Pretschner ; Sebastian Voss ; Michel Chaudron ; Jorg Kienzle ; Markus Volter ; Sebastien Gerard ; Mansooreh Zahedi ; Erwan Bousse ; Arend Rensink ; Fiona Polack ; Gregor Engels ; Gerti Kappel. IEEE, 2019. pp. 624-628
@inproceedings{4a5930cc72ff41cf8c238ea24cfe3d0e,
title = "Towards Model-Driven Self-Explanation for Autonomous Decision-Making Systems",
abstract = "The ability for systems to make decisions by themselves is increasing with advances in different areas of AI such as machine learning and optimisation techniques for autonomous systems among other. Humans are handing over more decisions to systems that provide no explanations for their judgements unless they are enabled explicitly in their design. Trust based on a program being well written and tested correctly is not appropriate for AI-based autonomous systems. Unlike traditional software, this new software increasingly exhibit emergent behaviours making it unpredictable due to unexpected situations. Self-explanation is sometimes implemented, tracking decisions to give explanations to users. A more consistent, proven approach to self-explanation would be needed for making trustable systems. The paper proposes a research agenda to define an architecture to enable self-explanation for autonomous decision-making systems. The approach will be model-driven to facilitate reuse, the rapid development of tools and suitable abstractions for demonstrating concepts. The architecture will be informed by existing research in provenance ontology and model version research. The evaluation of the architecture is expected to be done using two case studies. The first will implement self-explanation as a primary concern in the building of a system. The second case will attempt to fit self-explanation to an existing system.",
keywords = "Autonomous, Decision-making, Model-driven, Self-explanation",
author = "Owen Reynolds",
year = "2019",
month = "11",
day = "21",
doi = "10.1109/MODELS-C.2019.00095",
language = "English",
isbn = "978-1-7281-5126-7",
pages = "624--628",
editor = "Loli Burgueno and Loli Burgueno and Alexander Pretschner and Sebastian Voss and Michel Chaudron and Jorg Kienzle and Markus Volter and Sebastien Gerard and Mansooreh Zahedi and Erwan Bousse and Arend Rensink and Fiona Polack and Gregor Engels and Gerti Kappel",
booktitle = "Proceedings - 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2019",
publisher = "IEEE",
address = "United States",

}

Reynolds, O 2019, Towards Model-Driven Self-Explanation for Autonomous Decision-Making Systems. in L Burgueno, L Burgueno, A Pretschner, S Voss, M Chaudron, J Kienzle, M Volter, S Gerard, M Zahedi, E Bousse, A Rensink, F Polack, G Engels & G Kappel (eds), Proceedings - 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2019., 8904475, IEEE, pp. 624-628, 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), 15/09/19. https://doi.org/10.1109/MODELS-C.2019.00095

Towards Model-Driven Self-Explanation for Autonomous Decision-Making Systems. / Reynolds, Owen.

Proceedings - 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2019. ed. / Loli Burgueno; Loli Burgueno; Alexander Pretschner; Sebastian Voss; Michel Chaudron; Jorg Kienzle; Markus Volter; Sebastien Gerard; Mansooreh Zahedi; Erwan Bousse; Arend Rensink; Fiona Polack; Gregor Engels; Gerti Kappel. IEEE, 2019. p. 624-628 8904475.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Towards Model-Driven Self-Explanation for Autonomous Decision-Making Systems

AU - Reynolds, Owen

PY - 2019/11/21

Y1 - 2019/11/21

N2 - The ability for systems to make decisions by themselves is increasing with advances in different areas of AI such as machine learning and optimisation techniques for autonomous systems among other. Humans are handing over more decisions to systems that provide no explanations for their judgements unless they are enabled explicitly in their design. Trust based on a program being well written and tested correctly is not appropriate for AI-based autonomous systems. Unlike traditional software, this new software increasingly exhibit emergent behaviours making it unpredictable due to unexpected situations. Self-explanation is sometimes implemented, tracking decisions to give explanations to users. A more consistent, proven approach to self-explanation would be needed for making trustable systems. The paper proposes a research agenda to define an architecture to enable self-explanation for autonomous decision-making systems. The approach will be model-driven to facilitate reuse, the rapid development of tools and suitable abstractions for demonstrating concepts. The architecture will be informed by existing research in provenance ontology and model version research. The evaluation of the architecture is expected to be done using two case studies. The first will implement self-explanation as a primary concern in the building of a system. The second case will attempt to fit self-explanation to an existing system.

AB - The ability for systems to make decisions by themselves is increasing with advances in different areas of AI such as machine learning and optimisation techniques for autonomous systems among other. Humans are handing over more decisions to systems that provide no explanations for their judgements unless they are enabled explicitly in their design. Trust based on a program being well written and tested correctly is not appropriate for AI-based autonomous systems. Unlike traditional software, this new software increasingly exhibit emergent behaviours making it unpredictable due to unexpected situations. Self-explanation is sometimes implemented, tracking decisions to give explanations to users. A more consistent, proven approach to self-explanation would be needed for making trustable systems. The paper proposes a research agenda to define an architecture to enable self-explanation for autonomous decision-making systems. The approach will be model-driven to facilitate reuse, the rapid development of tools and suitable abstractions for demonstrating concepts. The architecture will be informed by existing research in provenance ontology and model version research. The evaluation of the architecture is expected to be done using two case studies. The first will implement self-explanation as a primary concern in the building of a system. The second case will attempt to fit self-explanation to an existing system.

KW - Autonomous

KW - Decision-making

KW - Model-driven

KW - Self-explanation

UR - https://ieeexplore.ieee.org/document/8904475/

UR - http://www.scopus.com/inward/record.url?scp=85075932569&partnerID=8YFLogxK

U2 - 10.1109/MODELS-C.2019.00095

DO - 10.1109/MODELS-C.2019.00095

M3 - Conference contribution

SN - 978-1-7281-5126-7

SP - 624

EP - 628

BT - Proceedings - 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2019

A2 - Burgueno, Loli

A2 - Burgueno, Loli

A2 - Pretschner, Alexander

A2 - Voss, Sebastian

A2 - Chaudron, Michel

A2 - Kienzle, Jorg

A2 - Volter, Markus

A2 - Gerard, Sebastien

A2 - Zahedi, Mansooreh

A2 - Bousse, Erwan

A2 - Rensink, Arend

A2 - Polack, Fiona

A2 - Engels, Gregor

A2 - Kappel, Gerti

PB - IEEE

ER -

Reynolds O. Towards Model-Driven Self-Explanation for Autonomous Decision-Making Systems. In Burgueno L, Burgueno L, Pretschner A, Voss S, Chaudron M, Kienzle J, Volter M, Gerard S, Zahedi M, Bousse E, Rensink A, Polack F, Engels G, Kappel G, editors, Proceedings - 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2019. IEEE. 2019. p. 624-628. 8904475 https://doi.org/10.1109/MODELS-C.2019.00095