ThermalYOLO: A Person Detection Neural Network in Thermal Images for Smart Environments

Nowadays, low-resolution thermal cameras are gaining relevance in smart environments due to keeping user privacy by recording images and videos in domestic environments. Many neural networks obtain outstanding results from visible spectrum devices for human activity and event detection, such as fall detection, object detection or pose estimation. However, these state-of-the-art neural networks are trained in datasets that do not contain thermal images, so their performance on them is not good. The main objective of this work is human body recognition and segmentation from thermal cameras. For this purpose, we propose ThermalYOLO, a neural network based on the YOLO neural network and fine-tuned with thermal images. For the generation and auto-labelling of the thermal dataset, an IoT device with two cameras, a visible camera and a thermal camera, is used. Therefore, the user does not have to manually annotate the dataset. As a result, ThermalYOLO outperforms YOLO in thermal images from two different smart environments.