Towards multimodal affective expression: merging facial expressions and body motion into emotion

Diego R. Faria, Fernanda C. C. Faria, Cristiano Premebida

Research output: Chapter in Book/Published conference outputConference publication


Affect recognition plays an important role in human everyday life and it is a substantial way of communication through expressions. Humans can rely on different channels of information to understand the affective messages communicated with others. Similarly, it is expected that an automatic affect
recognition system should be able to analyse different types of emotion expressions. In this respect, an important issue to be addressed is the fusion of different channels of expression, taking into account the relationship and correlation across different modalities. In this work, affective facial and bodily
motion expressions are addressed as channels for the communication of affect, designed as an emotion recognition system. A probabilistic approach is used to combine features from two modalities by incorporating geometric facial expression features and body motion skeleton-based features. Preliminary
results show that the presented approach has potential for automatic emotion recognition and it can be used for human robot interaction.
Original languageEnglish
Title of host publicationIEEE RO-MAN'17: Workshop Proceedings on Artificial Perception, Machine Learning and Datasets for Human-Robot Interaction (ARMADA'17), pp.16-20.
Number of pages5
Publication statusPublished - 18 Sept 2017

Bibliographical note

Copyright: IEEE & ARMADA 2017


  • Emotion recognition
  • probabilistic approach
  • human-robot interaction


Dive into the research topics of 'Towards multimodal affective expression: merging facial expressions and body motion into emotion'. Together they form a unique fingerprint.

Cite this