Multi-scale Optical Flow Estimation for Video Infrared Small Target Detection

The spatio-temporal information among video sequences is significant for video infrared (IR) small target detection. To effectively utilize the supplementary temporal information, existing video IR small target detection methods usually use optical flow to perform motion estimation and compensation. The common optical flow-based detection methods can only capture small motion of video sequences. However, the slow IR imaging speed and wide viewing distance resulting the spatial location of the target between two frames is frequently different, which limits the efficacy of optical flow-based detection methods. To solve the problem, we propose an end-to-end video infrared small target detection method, which is more robust to large motion and can achieve more accurate motion compensation. Specifically, we first propose a multi-scale optical flow reconstruction network to perform motion estimation in a coarse-to-fine manner. Then, the generated optical flows are used to align the neighborhood frames to the reference frame. Finally, the aligned neighborhood frames are concatenated and fed to the detection network to generate detection results. Experimental results demonstrate that the proposed reconstruction network provides more accurate motion estimation and can achieve improved performance on both detection accuracy and temporal consistency. Comparative results on the infrared video datasets show that our framework achieves the state-of-the-art performance.