Context-aware pedestrian intent prediction for connected and automated vehicles

Abstract

The safety of pedestrians crossing intersections continues to raise concerns since a continuous level of awareness should be maintained to achieve optimal safety. Advanced safety features in assisted and self-driving vehicles have the potential to augment situational awareness and improve the safety of road users. Detecting pedestrian intention is a key element in providing safer urban environments to automated vehicles. However, the complexity involved in anticipating pedestrian crossing intentions makes the task challenging, as these are internal states characterized by dynamic, non-verbal signals, and unpredictable or sudden movements. Such intricacies can lead to misunderstandings for autonomous vehicles, potentially leading to a higher incidence of pedestrian accidents. To achieve this target, there is a need for real-time crossing intentions for pedestrians. In recent years, there has been an increasing shift towards a new trend in extending deep neural networks into non-Euclidean spaces, commonly known as geometric deep learning. Research in deep learning on graphs is gaining momentum, showcasing the powerful descriptive capabilities of graph structures. These structures provide essential relationship data among various elements, proving invaluable across diverse learning applications. Our study introduces an innovative spatio-temporal graph-based model designed to extract and leverage spatial and temporal features from spatio-temporal graph data. The goal is to uncover hidden patterns within these graphs, which is crucial for accurately predicting pedestrian crossing intentions. By integrating both visual and contextual information, we present a unique approach to structuring traffic scenes as graph data. This involves a novel ego-centric vehicle approach with a wheel topology that effectively captures the dynamics and interactions among traffic participants, the surrounding environment, and pedestrians. Additionally, we explore the most impactful nodes that significantly influence the model’s predictive accuracy. The model was tested on three datasets, demonstrating promising results in predicting pedestrian crossing intentions across various urban traffic scenarios.