Visual Vehicle Tracking Through Noise and Occlusions Using Crowd-Sourced Maps

We present a location-specific method to visually track the positions of observed vehicles based on large-scale crowd-sourced maps. We equipped a large fleet of cars that drive around cities with camera phones mounted on the dashboard, and performed city-scale structure-from-motion to accurately reconstruct the trajectories taken by the vehicles. We show that these data can be used to first create a system enabling high-accuracy localisation, and then to accurately predict the future motion of newly observed cars in the camera view. As a basis for the method we use a recently proposed system [1] for unsupervised motion prediction and extend it to a real-time visual tracking pipeline which can track vehicles through noise and extended occlusions using only a monocular camera. The system is tested using two large-scale datasets of San Francisco and New York City containing millions of frames. We demonstrate the performance of the system in a variety of traffic, time, and weather conditions. The presented system requires no manual annotation or knowledge of road infrastructure. To our knowledge, this is the first time a perception system based on a large-scale crowd-sourced maps has been evaluated at this scale.