Illumination-insensitive background subtraction

A core challenge in background subtraction (BGS) is handling videos with sudden illumination changes in consecutive frames. While the use of data augmentation has been shown to increase robustness, the modelling of realistic illumination changes remains less explored and is usually limited to global, static brightness adjustments. In this paper, we focus on tackling the problem of background subtraction using augmented training data, and propose an augmentation method which vastly improves the model’s performance under challenging illumination conditions. In particular, our framework consists of a local component that considers direct light/shadow and lighting angles, and a global component that considers the overall contrast, sharpness and color saturation of the image. It generates realistic, structured training data with different illumination conditions, enabling our deep learning system to be trained effectively for background subtraction even when significant illumination changes take place. We further propose a post-processing method that removes noise from the output binary map of segmentation, resulting in a cleaner, more accurate segmentation map that can generalise to multiple scenes of different conditions. Experimental results demonstrate that the proposed system outperforms existing work, with the highest F-measure score of 81.27% obtained by the full system. To facilitate the research in the field, we open the source code of this project at: https://github.com/dksakkos/illumination_augmentation