Social Activity Recognition on Continuous RGB-D Video Sequences

Claudio Coppola*, Serhan Cosar, Diego R. Faria, Nicola Bellotto

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

Modern service robots are provided with one or more sensors, often including RGB-D cameras, to perceive objects and humans in the environment. This paper proposes a new system for the recognition of human social activities from a continuous stream of RGB-D data. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications it is important to be able to move to more realistic scenarios in which such activities are not manually selected. For this reason, it is useful to detect the time intervals when humans are performing social activities, the recognition of which can contribute to trigger human-robot interactions or to detect situations of potential danger. The main contributions of this research work include a novel system for the recognition of social activities from continuous RGB-D data, combining temporal segmentation and classification, as well as a model for learning the proximity-based priors of the social activities. A new public dataset with RGB-D videos of social and individual activities is also provided and used for evaluating the proposed solutions. The results show the good performance of the system in recognising social activities from continuous RGB-D data.

Original languageEnglish
Number of pages15
JournalInternational Journal of Social Robotics
Early online date29 Apr 2019
DOIs
Publication statusE-pub ahead of print - 29 Apr 2019

Fingerprint

Human robot interaction
Robotics
Cameras
Robots
Sensors

Bibliographical note

© The Author(s) 2019. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Keywords

  • Activity recognition
  • Activity temporal segmentation
  • Machine learning
  • Social activity recognition

Cite this

Coppola, C., Cosar, S., Faria, D. R., & Bellotto, N. (2019). Social Activity Recognition on Continuous RGB-D Video Sequences. International Journal of Social Robotics. https://doi.org/10.1007/s12369-019-00541-y
Coppola, Claudio ; Cosar, Serhan ; Faria, Diego R. ; Bellotto, Nicola. / Social Activity Recognition on Continuous RGB-D Video Sequences. In: International Journal of Social Robotics. 2019.
@article{9683d0e357244f20b24497a67547dc79,
title = "Social Activity Recognition on Continuous RGB-D Video Sequences",
abstract = "Modern service robots are provided with one or more sensors, often including RGB-D cameras, to perceive objects and humans in the environment. This paper proposes a new system for the recognition of human social activities from a continuous stream of RGB-D data. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications it is important to be able to move to more realistic scenarios in which such activities are not manually selected. For this reason, it is useful to detect the time intervals when humans are performing social activities, the recognition of which can contribute to trigger human-robot interactions or to detect situations of potential danger. The main contributions of this research work include a novel system for the recognition of social activities from continuous RGB-D data, combining temporal segmentation and classification, as well as a model for learning the proximity-based priors of the social activities. A new public dataset with RGB-D videos of social and individual activities is also provided and used for evaluating the proposed solutions. The results show the good performance of the system in recognising social activities from continuous RGB-D data.",
keywords = "Activity recognition, Activity temporal segmentation, Machine learning, Social activity recognition",
author = "Claudio Coppola and Serhan Cosar and Faria, {Diego R.} and Nicola Bellotto",
note = "{\circledC} The Author(s) 2019. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.",
year = "2019",
month = "4",
day = "29",
doi = "10.1007/s12369-019-00541-y",
language = "English",

}

Coppola, C, Cosar, S, Faria, DR & Bellotto, N 2019, 'Social Activity Recognition on Continuous RGB-D Video Sequences', International Journal of Social Robotics. https://doi.org/10.1007/s12369-019-00541-y

Social Activity Recognition on Continuous RGB-D Video Sequences. / Coppola, Claudio; Cosar, Serhan; Faria, Diego R.; Bellotto, Nicola.

In: International Journal of Social Robotics, 29.04.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Social Activity Recognition on Continuous RGB-D Video Sequences

AU - Coppola, Claudio

AU - Cosar, Serhan

AU - Faria, Diego R.

AU - Bellotto, Nicola

N1 - © The Author(s) 2019. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

PY - 2019/4/29

Y1 - 2019/4/29

N2 - Modern service robots are provided with one or more sensors, often including RGB-D cameras, to perceive objects and humans in the environment. This paper proposes a new system for the recognition of human social activities from a continuous stream of RGB-D data. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications it is important to be able to move to more realistic scenarios in which such activities are not manually selected. For this reason, it is useful to detect the time intervals when humans are performing social activities, the recognition of which can contribute to trigger human-robot interactions or to detect situations of potential danger. The main contributions of this research work include a novel system for the recognition of social activities from continuous RGB-D data, combining temporal segmentation and classification, as well as a model for learning the proximity-based priors of the social activities. A new public dataset with RGB-D videos of social and individual activities is also provided and used for evaluating the proposed solutions. The results show the good performance of the system in recognising social activities from continuous RGB-D data.

AB - Modern service robots are provided with one or more sensors, often including RGB-D cameras, to perceive objects and humans in the environment. This paper proposes a new system for the recognition of human social activities from a continuous stream of RGB-D data. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications it is important to be able to move to more realistic scenarios in which such activities are not manually selected. For this reason, it is useful to detect the time intervals when humans are performing social activities, the recognition of which can contribute to trigger human-robot interactions or to detect situations of potential danger. The main contributions of this research work include a novel system for the recognition of social activities from continuous RGB-D data, combining temporal segmentation and classification, as well as a model for learning the proximity-based priors of the social activities. A new public dataset with RGB-D videos of social and individual activities is also provided and used for evaluating the proposed solutions. The results show the good performance of the system in recognising social activities from continuous RGB-D data.

KW - Activity recognition

KW - Activity temporal segmentation

KW - Machine learning

KW - Social activity recognition

UR - http://www.scopus.com/inward/record.url?scp=85065185036&partnerID=8YFLogxK

UR - https://link.springer.com/article/10.1007%2Fs12369-019-00541-y

U2 - 10.1007/s12369-019-00541-y

DO - 10.1007/s12369-019-00541-y

M3 - Article

AN - SCOPUS:85065185036

ER -

Coppola C, Cosar S, Faria DR, Bellotto N. Social Activity Recognition on Continuous RGB-D Video Sequences. International Journal of Social Robotics. 2019 Apr 29. https://doi.org/10.1007/s12369-019-00541-y