Highlight-assisted Nighttime Vehicle Detection Using a Multi-level Fusion Network and Label Hierarchy

Nighttime vehicle detection plays an essential role in Automatic Driving System (ADS) and Driver Assistance System (DAS). However, the visual features of the vehicle at night are undistinguishable due to low illumination and may bring significant challenges to the vision-based method. Vehicle highlight information including vehicle lights and the correspondingly reflected lights is high-confident visual features at night. Thus a detection system effectively using vehicle highlight information can gain large improvements. In this paper, we propose a novel nighttime vehicle detection framework with assistance from the vehicle highlight information. Firstly, we generate a fine-grained vehicle highlight detector and create the vehicle label hierarchy to enlarge the inter-class difference and reduce the intra-class gap. Then, we propose a feature aggregation mechanism to combine multi-scale highlight features and the vehicle’s visual features to take advantages of the vehicle highlight position hints. With the novel vehicle highlight aggregation mechanism, the performance of our method goes beyond that of the state-of-the-art. Also, our method has gained improvements when transferring our method to mainstream frameworks, indicating that our approach is both effective and general.