AI-based Pedestrian Detection and Avoidance at Night using an IR Camera, Radar, and a Video Camera

In 2019, the United States experienced more than 6,500 pedestrian fatalities involving motor vehicles which resulted in a 67% rise in nighttime pedestrian fatalities and only a 10% rise in daytime pedestrian fatalities. In an effort to reduce fatalities, this research developed a pedestrian detection and alert system through the application of a visual camera, infrared camera, and radar sensors combined with machine learning. The research team designed the system concept to achieve a high level of accuracy in pedestrian detection and avoidance during both the day and at night to avoid potentially fatal accidents involving pedestrians crossing a street. The working prototype of pedestrian detection and collision avoidance can be installed in present-day vehicles, with the visible camera used to detect pedestrians during the day and the infrared camera to detect pedestrians primarily during the night as well as at high glare from the sun during the day. The radar sensor is also used to detect the presence of a pedestrian and calculate their range and direction of motion relative to the vehicle. Through data fusion and deep learning, the ability to quickly analyze and classify a pedestrian’s presence at all times in a real-time monitoring system is achieved. The system can also be extended to cyclist and animal detection and avoidance, and could be deployed in an autonomous vehicle to assist in automatic braking systems (ABS).