Mobile Crowd Sensing Based Road Anomaly Detection using Mask R-CNN

Abstract Road anomalies such as potholes and uneven surfaces create accident-prone zones that should be properly maintained to develop an easily scalable and intelligent transportation system. In this context, this study presents a unique contribution in the form of an artificial intelligence (AI)-powered mobile crowd sensing system for efficient detection, monitoring and mapping of road abnormalities. Specifically, an approach is proposed to build an end-to-end mobile application with a deep learning based object detection model in the cloud that can localize and classify road anomalies in images and videos. The models including Mask Region Convolutional Neural Networks (MRCNN), You Look Only Once (YOLO) and Single Shot Detector (SSD) are pre-trained using the COCO dataset, followed by application of transfer learning to retrain them on a road anomalies dataset. This dataset contains a pool of 1400 pre-processed, augmented images of potholes, uneven surfaces, manholes and speed bumps. The performance of the models are evaluated on the basis of established object detection evaluation parameters like Intersection over Union (IoU), mean Average Precision (mAP), log-average miss rate (LAMR) and model detection frequency in frames per second (fps). Two stage detector Mask R-CNN, specifically with ResNet101 backbone performs better in terms of overall mAP and LAMR scores, while YOLO-v3 performs the fastest. The proposed model is then used in a mobile application which will allow users to take images or videos of roads. These will be processed in the server and detected road anomalies are shown on a global map to assist in better road infrastructure and transportation systems.