Analysis of abnormal pedestrian behaviors at grade crossings based on semi-supervised generative adversarial networks

Unexpected intrusions by pedestrians at grade crossings pose significant risks to the safety of railroad operations. Currently, there is no information processing system available for monitoring anomalies at grade crossings. Therefore, this paper presents a video processing pipeline and a generative adversarial network (GAN)-based deep learning framework to detect, localize, and analyze abnormal behaviors of pedestrians at grade crossings. First, the motions of pedestrians are represented by temporally-varying trajectories of key points identified by skeleton detection and tracking algorithms. A GAN model is developed to learn both global and local motion features of normal pedestrians only. The abnormal behaviors can then be detected as outliers by a discriminator during the testing phase. In contrast to existing efforts, several measures are taken to further boost model performance, including purposely exploiting overfitting in training to magnify the difference between the normal and the abnormal motion patterns and adding an appropriate amount of noise to enhance model generalization. The experiments conducted on a custom video dataset demonstrate the remarkable performance of our model, which successfully distinguishes motion patterns of squatting and lingering from normal walking. The model achieves a value of 0.89 in the AUC (Area Under the Curve) and notably outperforms the other seven benchmark models. The present method is also able to analyze multiple pedestrians in one video frame with a single run of the GAN model and requires no location-specific information, enabling salient robustness and field-deployability of the model at different locations without retraining.