An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence

Panagiotis D. Bamidis, Christos A. Frantzidis, Evdokimos I. Konstantinidis, Andrej Luneski, Chrysa Lithari, Manousos A. Klados, Charalambos Bratsas, Christos L. Papadelis, Costas Pappas

Research output: Chapter in Book/Report/Conference proceedingOther chapter contribution

Abstract

Emotion identification is beginning to be considered as an essential feature in human-computer interaction. However, most of the studies are mainly focused on facial expression classifications and speech recognition and not much attention has been paid until recently to physiological pattern recognition. In this paper, an integrative approach is proposed to emotional interaction by fusing multi-modal signals. Subjects are exposed to pictures selected from the International Affective Picture System (IAPS). A feature extraction procedure is used to discriminate between four affective states by means of a Mahalanobis distance classifier. The average classifications rate (74.11%) was encouraging. Thus, the induced affective state is mirrored through an avatar by changing its facial characteristics and generating a voice message sympathising with the user’s mood. It is argued that multi-physiological patterning in combination with anthropomorphic avatars may contribute to the enhancement of affective multi-modal interfaces and the advancement of machine emotional intelligence.
LanguageEnglish
Title of host publicationHCI 2009: Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction
PublisherSpringer
Pages565-574
ISBN (Electronic)978-3-642-02580-8
ISBN (Print)978-3-642-02579-2
DOIs
Publication statusPublished - 2009

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume5612
ISSN (Print)0302-9743

Fingerprint

Human computer interaction
Speech recognition
Pattern recognition
Feature extraction
Classifiers

Cite this

Bamidis, P. D., Frantzidis, C. A., Konstantinidis, E. I., Luneski, A., Lithari, C., Klados, M. A., ... Pappas, C. (2009). An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence. In HCI 2009: Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction (pp. 565-574). [Chapter 62] (Lecture Notes in Computer Science; Vol. 5612). Springer. https://doi.org/10.1007/978-3-642-02580-8_62
Bamidis, Panagiotis D. ; Frantzidis, Christos A. ; Konstantinidis, Evdokimos I. ; Luneski, Andrej ; Lithari, Chrysa ; Klados, Manousos A. ; Bratsas, Charalambos ; Papadelis, Christos L. ; Pappas, Costas. / An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence. HCI 2009: Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction. Springer, 2009. pp. 565-574 (Lecture Notes in Computer Science).
@inbook{bc9aadebead0480291bb231095640cc1,
title = "An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence",
abstract = "Emotion identification is beginning to be considered as an essential feature in human-computer interaction. However, most of the studies are mainly focused on facial expression classifications and speech recognition and not much attention has been paid until recently to physiological pattern recognition. In this paper, an integrative approach is proposed to emotional interaction by fusing multi-modal signals. Subjects are exposed to pictures selected from the International Affective Picture System (IAPS). A feature extraction procedure is used to discriminate between four affective states by means of a Mahalanobis distance classifier. The average classifications rate (74.11{\%}) was encouraging. Thus, the induced affective state is mirrored through an avatar by changing its facial characteristics and generating a voice message sympathising with the user’s mood. It is argued that multi-physiological patterning in combination with anthropomorphic avatars may contribute to the enhancement of affective multi-modal interfaces and the advancement of machine emotional intelligence.",
author = "Bamidis, {Panagiotis D.} and Frantzidis, {Christos A.} and Konstantinidis, {Evdokimos I.} and Andrej Luneski and Chrysa Lithari and Klados, {Manousos A.} and Charalambos Bratsas and Papadelis, {Christos L.} and Costas Pappas",
year = "2009",
doi = "10.1007/978-3-642-02580-8_62",
language = "English",
isbn = "978-3-642-02579-2",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "565--574",
booktitle = "HCI 2009: Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction",
address = "Germany",

}

Bamidis, PD, Frantzidis, CA, Konstantinidis, EI, Luneski, A, Lithari, C, Klados, MA, Bratsas, C, Papadelis, CL & Pappas, C 2009, An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence. in HCI 2009: Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction., Chapter 62, Lecture Notes in Computer Science, vol. 5612, Springer, pp. 565-574. https://doi.org/10.1007/978-3-642-02580-8_62

An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence. / Bamidis, Panagiotis D.; Frantzidis, Christos A.; Konstantinidis, Evdokimos I.; Luneski, Andrej; Lithari, Chrysa; Klados, Manousos A.; Bratsas, Charalambos; Papadelis, Christos L.; Pappas, Costas.

HCI 2009: Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction. Springer, 2009. p. 565-574 Chapter 62 (Lecture Notes in Computer Science; Vol. 5612).

Research output: Chapter in Book/Report/Conference proceedingOther chapter contribution

TY - CHAP

T1 - An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence

AU - Bamidis, Panagiotis D.

AU - Frantzidis, Christos A.

AU - Konstantinidis, Evdokimos I.

AU - Luneski, Andrej

AU - Lithari, Chrysa

AU - Klados, Manousos A.

AU - Bratsas, Charalambos

AU - Papadelis, Christos L.

AU - Pappas, Costas

PY - 2009

Y1 - 2009

N2 - Emotion identification is beginning to be considered as an essential feature in human-computer interaction. However, most of the studies are mainly focused on facial expression classifications and speech recognition and not much attention has been paid until recently to physiological pattern recognition. In this paper, an integrative approach is proposed to emotional interaction by fusing multi-modal signals. Subjects are exposed to pictures selected from the International Affective Picture System (IAPS). A feature extraction procedure is used to discriminate between four affective states by means of a Mahalanobis distance classifier. The average classifications rate (74.11%) was encouraging. Thus, the induced affective state is mirrored through an avatar by changing its facial characteristics and generating a voice message sympathising with the user’s mood. It is argued that multi-physiological patterning in combination with anthropomorphic avatars may contribute to the enhancement of affective multi-modal interfaces and the advancement of machine emotional intelligence.

AB - Emotion identification is beginning to be considered as an essential feature in human-computer interaction. However, most of the studies are mainly focused on facial expression classifications and speech recognition and not much attention has been paid until recently to physiological pattern recognition. In this paper, an integrative approach is proposed to emotional interaction by fusing multi-modal signals. Subjects are exposed to pictures selected from the International Affective Picture System (IAPS). A feature extraction procedure is used to discriminate between four affective states by means of a Mahalanobis distance classifier. The average classifications rate (74.11%) was encouraging. Thus, the induced affective state is mirrored through an avatar by changing its facial characteristics and generating a voice message sympathising with the user’s mood. It is argued that multi-physiological patterning in combination with anthropomorphic avatars may contribute to the enhancement of affective multi-modal interfaces and the advancement of machine emotional intelligence.

UR - https://link.springer.com/chapter/10.1007/978-3-642-02580-8_62

U2 - 10.1007/978-3-642-02580-8_62

DO - 10.1007/978-3-642-02580-8_62

M3 - Other chapter contribution

SN - 978-3-642-02579-2

T3 - Lecture Notes in Computer Science

SP - 565

EP - 574

BT - HCI 2009: Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction

PB - Springer

ER -

Bamidis PD, Frantzidis CA, Konstantinidis EI, Luneski A, Lithari C, Klados MA et al. An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence. In HCI 2009: Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction. Springer. 2009. p. 565-574. Chapter 62. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-642-02580-8_62