Multi-scale pedestrian intent prediction using 3D joint information as spatio-temporal representation

Sarfraz Ahmed, Ammar Al-Bazi, Chitta Saha, Sujan Rajbhandari, M. Nazmul Huda*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


There has been a rise of use of Autonomous Vehicles on public roads. With the predicted rise of road traffic accidents over the coming years, these vehicles must be capable of safely operate in the public domain. The field of pedestrian detection has significantly advanced in the last decade, providing high-level accuracy, with some technique reaching near-human level accuracy. However, there remains further work required for pedestrian intent prediction to reach human-level performance. One of the challenges facing current pedestrian intent predictors are the varying scales of pedestrians, particularly smaller pedestrians. This is because smaller pedestrians can blend into the background, making them difficult to detect, track or apply pose estimations techniques. Therefore, in this work, we present a novel intent prediction approach for multi-scale pedestrians using 2D pose estimation and a Long Short-term memory (LSTM) architecture. The pose estimator predicts keypoints for the pedestrian along the video frames. Based on the accumulation of these keypoints along the frames, spatio-temporal data is generated. This spatio-temporal data is fed to the LSTM for classifying the crossing behaviour of the pedestrians. We evaluate the performance of the proposed techniques on the popular Joint Attention in Autonomous Driving (JAAD) dataset and the new larger-scale Pedestrian Intention Estimation (PIE) dataset. Using data generalisation techniques, we show that the proposed technique outperformed the state-of-the-art techniques by up to 7%, reaching up to 94% accuracy while maintaining a comparable run-time of 6.1 ms.
Original languageEnglish
Article number120077
Number of pages11
JournalExpert Systems with Applications
Early online date13 Apr 2023
Publication statusPublished - 1 Sept 2023

Bibliographical note

Copyright © 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (


  • Intent prediction
  • LSTM
  • Pedestrian detection
  • Pose estimation
  • Tracking


Dive into the research topics of 'Multi-scale pedestrian intent prediction using 3D joint information as spatio-temporal representation'. Together they form a unique fingerprint.

Cite this