Small Flying Object Detection and Tracking in Digital Airport Tower through Spatial-Temporal ConvNets

Abstract Digital towers using high-resolution cameras that cover a 360-degree view of airports have recently been applied as a solution for some airports by replacing conventional towers. Although many computer vision systems have been developed as tools to assist tower controllers, small flying object detection remains challenging due to their small dimensions and unpredictable trajectories. This paper proposes a novel computer vision framework to detect, track and recognize small flying objects, namely aircraft and drones, in an airport environment. The framework creates a new Convolutional Neural Network which adapts to the unique characteristics of small flying objects. It also exploits the spatial-temporal information, as well as post-processing, to improve the performance. The proposed framework is validated on an airport dataset and Drone-vs-Bird public dataset. The results show that the framework can not only perform object detection in real-time, but also surpass the performance of state-of-the-art models in both datasets by a large margin.