Towards multimodal affective expression: merging facial expressions and body motion into emotion

Diego R. Faria; Fernanda C. C. Faria; Cristiano Premebida

Towards multimodal affective expression: merging facial expressions and body motion into emotion

Diego R. Faria, Fernanda C. C. Faria, Cristiano Premebida

College of Engineering and Physical Sciences

Research output: Chapter in Book/Published conference output › Conference publication

Abstract

Affect recognition plays an important role in human everyday life and it is a substantial way of communication through expressions. Humans can rely on different channels of information to understand the affective messages communicated with others. Similarly, it is expected that an automatic affect
recognition system should be able to analyse different types of emotion expressions. In this respect, an important issue to be addressed is the fusion of different channels of expression, taking into account the relationship and correlation across different modalities. In this work, affective facial and bodily
motion expressions are addressed as channels for the communication of affect, designed as an emotion recognition system. A probabilistic approach is used to combine features from two modalities by incorporating geometric facial expression features and body motion skeleton-based features. Preliminary
results show that the presented approach has potential for automatic emotion recognition and it can be used for human robot interaction.

Original language	English
Title of host publication	IEEE RO-MAN'17: Workshop Proceedings on Artificial Perception, Machine Learning and Datasets for Human-Robot Interaction (ARMADA'17), pp.16-20.
Publisher	IEEE
Pages	16-20
Number of pages	5
Publication status	Published - 18 Sept 2017

Bibliographical note

Copyright: IEEE & ARMADA 2017

Keywords

Emotion recognition
probabilistic approach
human-robot interaction

Access to Document

Towards Multimodal Affective Expression
Copyright: IEEE & ARMADA 2017.
Accepted author manuscript, 1.4 MB

https://sites.google.com/site/roman17armada/home

Cite this

@inproceedings{0fb9a6a7b29f461ca14a265c75d43553,

title = "Towards multimodal affective expression: merging facial expressions and body motion into emotion",

abstract = "Affect recognition plays an important role in human everyday life and it is a substantial way of communication through expressions. Humans can rely on different channels of information to understand the affective messages communicated with others. Similarly, it is expected that an automatic affectrecognition system should be able to analyse different types of emotion expressions. In this respect, an important issue to be addressed is the fusion of different channels of expression, taking into account the relationship and correlation across different modalities. In this work, affective facial and bodilymotion expressions are addressed as channels for the communication of affect, designed as an emotion recognition system. A probabilistic approach is used to combine features from two modalities by incorporating geometric facial expression features and body motion skeleton-based features. Preliminaryresults show that the presented approach has potential for automatic emotion recognition and it can be used for human robot interaction.",

keywords = "Emotion recognition, probabilistic approach, human-robot interaction",

author = "Faria, {Diego R.} and Faria, {Fernanda C. C.} and Cristiano Premebida",

note = "Copyright: IEEE & ARMADA 2017",

year = "2017",

month = sep,

day = "18",

language = "English",

pages = "16--20",

booktitle = "IEEE RO-MAN'17: Workshop Proceedings on Artificial Perception, Machine Learning and Datasets for Human-Robot Interaction (ARMADA'17), pp.16-20.",

publisher = "IEEE",

address = "United States",

}

Towards multimodal affective expression: merging facial expressions and body motion into emotion. / Faria, Diego R.; Faria, Fernanda C. C.; Premebida, Cristiano.
IEEE RO-MAN'17: Workshop Proceedings on Artificial Perception, Machine Learning and Datasets for Human-Robot Interaction (ARMADA'17), pp.16-20.. IEEE, 2017. p. 16-20.

Research output: Chapter in Book/Published conference output › Conference publication

TY - GEN

T1 - Towards multimodal affective expression

T2 - merging facial expressions and body motion into emotion

AU - Faria, Diego R.

AU - Faria, Fernanda C. C.

AU - Premebida, Cristiano

PY - 2017/9/18

Y1 - 2017/9/18

N2 - Affect recognition plays an important role in human everyday life and it is a substantial way of communication through expressions. Humans can rely on different channels of information to understand the affective messages communicated with others. Similarly, it is expected that an automatic affectrecognition system should be able to analyse different types of emotion expressions. In this respect, an important issue to be addressed is the fusion of different channels of expression, taking into account the relationship and correlation across different modalities. In this work, affective facial and bodilymotion expressions are addressed as channels for the communication of affect, designed as an emotion recognition system. A probabilistic approach is used to combine features from two modalities by incorporating geometric facial expression features and body motion skeleton-based features. Preliminaryresults show that the presented approach has potential for automatic emotion recognition and it can be used for human robot interaction.

AB - Affect recognition plays an important role in human everyday life and it is a substantial way of communication through expressions. Humans can rely on different channels of information to understand the affective messages communicated with others. Similarly, it is expected that an automatic affectrecognition system should be able to analyse different types of emotion expressions. In this respect, an important issue to be addressed is the fusion of different channels of expression, taking into account the relationship and correlation across different modalities. In this work, affective facial and bodilymotion expressions are addressed as channels for the communication of affect, designed as an emotion recognition system. A probabilistic approach is used to combine features from two modalities by incorporating geometric facial expression features and body motion skeleton-based features. Preliminaryresults show that the presented approach has potential for automatic emotion recognition and it can be used for human robot interaction.

KW - Emotion recognition

KW - probabilistic approach

KW - human-robot interaction

M3 - Conference publication

SP - 16

EP - 20

BT - IEEE RO-MAN'17: Workshop Proceedings on Artificial Perception, Machine Learning and Datasets for Human-Robot Interaction (ARMADA'17), pp.16-20.

PB - IEEE

ER -

Towards multimodal affective expression: merging facial expressions and body motion into emotion

Abstract

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this