Towards Model-Driven Self-Explanation for Autonomous Decision-Making Systems

The ability for systems to make decisions by themselves is increasing with advances in different areas of AI such as machine learning and optimisation techniques for autonomous systems among other. Humans are handing over more decisions to systems that provide no explanations for their judgements unless they are enabled explicitly in their design. Trust based on a program being well written and tested correctly is not appropriate for AI-based autonomous systems. Unlike traditional software, this new software increasingly exhibit emergent behaviours making it unpredictable due to unexpected situations. Self-explanation is sometimes implemented, tracking decisions to give explanations to users. A more consistent, proven approach to self-explanation would be needed for making trustable systems. The paper proposes a research agenda to define an architecture to enable self-explanation for autonomous decision-making systems. The approach will be model-driven to facilitate reuse, the rapid development of tools and suitable abstractions for demonstrating concepts. The architecture will be informed by existing research in provenance ontology and model version research. The evaluation of the architecture is expected to be done using two case studies. The first will implement self-explanation as a primary concern in the building of a system. The second case will attempt to fit self-explanation to an existing system.