Robot Multi-Modal Object Perception and Recognition: Synthetic Maturation of Sensorimotor Learning in Embodied Systems

Raphaël Braud, Alexandros Giagkos*, Patricia Shaw, Mark Lee, Qiang Shen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


It is known that during early infancy, humans experience many physical and cognitive changes that shape their learning and refine their understanding of objects in the world. With the extended arm being one of the very first objects they familiarise, infants undergo a series of developmental stages that progressively facilitate physical interactions, enrich sensory information and develop the skills to learn and recognise. Drawing inspiration from infancy, this study deals with the modelling of an open-ended learning mechanism for embodied agents that considers the cumulative and increasing complexity of physical interactions with the world. The proposed system achieves object perception, and recognition as the agent (i.e., a humanoid robot) matures, experiences changes to its visual capabilities, develops sensorimotor control, and interacts with objects within its reach. The reported findings demonstrate the critical role of developing vision on the effectiveness of object learning and recognition and the importance of reaching and grasping in solving visually elicited ambiguities. Impediments caused by the interdependency of parallel components responsible for the agent’s physical and cognitive functionalities are exposed, demonstrating an interesting phase transition in utilising object perceptions for recognition.
Original languageEnglish
Article number0
Number of pages13
JournalIEEE Transactions on Cognitive and Developmental Systems
Issue number0
Early online date13 Jan 2020
Publication statusE-pub ahead of print - 13 Jan 2020

Bibliographical note

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Funding: This research is supported by the UK Engineering and
Physical Sciences Research Council (EPSRC), grant No.


  • Multi-modal object learning
  • developmental learning
  • iCub robot.
  • longitudinal study
  • reaching
  • vision


Dive into the research topics of 'Robot Multi-Modal Object Perception and Recognition: Synthetic Maturation of Sensorimotor Learning in Embodied Systems'. Together they form a unique fingerprint.

Cite this