Social Activity Recognition on Continuous RGB-D Video Sequences

Claudio Coppola*, Serhan Cosar, Diego R. Faria, Nicola Bellotto

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Modern service robots are provided with one or more sensors, often including RGB-D cameras, to perceive objects and humans in the environment. This paper proposes a new system for the recognition of human social activities from a continuous stream of RGB-D data. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications it is important to be able to move to more realistic scenarios in which such activities are not manually selected. For this reason, it is useful to detect the time intervals when humans are performing social activities, the recognition of which can contribute to trigger human-robot interactions or to detect situations of potential danger. The main contributions of this research work include a novel system for the recognition of social activities from continuous RGB-D data, combining temporal segmentation and classification, as well as a model for learning the proximity-based priors of the social activities. A new public dataset with RGB-D videos of social and individual activities is also provided and used for evaluating the proposed solutions. The results show the good performance of the system in recognising social activities from continuous RGB-D data.

Original languageEnglish
Pages (from-to)201–215
Number of pages15
JournalInternational Journal of Social Robotics
Early online date29 Apr 2019
Publication statusPublished - Jan 2020

Bibliographical note

© The Author(s) 2019. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.


  • Activity recognition
  • Activity temporal segmentation
  • Machine learning
  • Social activity recognition


Dive into the research topics of 'Social Activity Recognition on Continuous RGB-D Video Sequences'. Together they form a unique fingerprint.

Cite this