Time-First Tracking: An Efficient Multiple-Object Tracking Architecture For Dynamic Surveillance Environments

Given the countless hours of video that are generated in surveillance environments, real-time for multi-object tracking (MOT) is vastly insufficient. Current MOT methods prioritize tracking accuracy in crowded environments, with little concern for total computational expense, which has led to a reliance on expensive object detectors to perform tracking. Indiscriminate use of object detectors is not scalable for surveillance problems and ignores the inherent spatio-temporal variation in scene complexity in many real-world environments. A novel MOT method is proposed, termed "Time-First Tracking", which relies on "shallowly" processed motion with a new tracking method, leaving the use of expensive object detection methods to an "as-needed" basis. The resulting vast reduction in pixels-processed may yield orders of magnitude in cost savings, making MOT more tractable. Time-First Tracking is adaptable to spatio-temporal changes in tracking difficulty; videos are divided into spatio-temporal sub-volumes, rated with different tracking difficulties, that are subsequently processed with different object localization methods. New MOT metrics are proposed to account for cost along with code to create a synthetic MOT dataset for motion-based tracking.