Emotion Recognition in Older Adults’ Speech: toward Empathetic Conversational AI in Ambient Assisted Living

Conversational AI systems are increasingly used to support older adults in their daily lives, particularly within Ambient Assisted Living (AAL) environments. While voice-based interaction offers a natural and accessible interface, current systems often lack emotional awareness, resulting in limited interaction quality and reduced perceived social presence. Enabling empathetic interactions through emotion recognition (ER) is therefore essential for improving user experience and long-term engagement. Speech provides a primary and non-intrusive modality for capturing emotional cues in conversational settings. However, existing speech emotion recognition (SER) models are predominantly trained on data from younger populations and often fail to generalise effectively to older adults’ speech due to age-related acoustic variations and data scarcity. Addressing this gap is critical for developing inclusive and robust conversational AI systems. In this study, we propose a two-stage transfer learning pipeline combining self-supervised speech representations with domain-adaptive fine-tuning, the first such approach specifically targeting ER in older adults’ speech within AAL environments. Pretrained models are adapted to the target domain using domain-adaptive fine-tuning to capture age-specific emotional and acoustic characteristics. The proposed approach is evaluated on the ElderReact dataset, a multimodal collection of recordings of older adults expressing emotions. Experimental results demonstrate that proposed pipeline significantly improve ER performance for older adults’ speech, with HuBERT-base achieving an F1 score of 0.72, a substantial improvement over non-adapted baselines. These findings highlight the effectiveness of self-supervised speech representations in low-resource settings and their potential to enable more empathetic conversational AI systems within AAL environments.