Intention prediction of pedestrians in challenging weather conditions using deep learning

Abstract

Assisted and automated driving vehicles have received massive attention over the past few years from the research community to make our roads safer. In this thesis, we introduce a framework for predicting the intention of pedestrians in clear and challenging weather conditions. The framework consists of five deep-learning models, of which two are designed and trained from scratch and three were used pretrained. The framework takes video frames from the dashcam and inputs them to an enhancement pipeline to determine the quality of the images and enhance them if necessary. Then, the framework utilizes pretrained models (MoveNet, Deep-sort, and Deep-Labv3) for feature extraction. Lastly, all the features are fed into a Transformer-based Intention Prediction Model (TIPM) for pedestrian intention prediction. Results show that TIPM outperforms state-of-the-art models yielding an accuracy of 69% on the JAAD behavior dataset, 82% on the JAAD all dataset.