Real-time tracking-with-detection for coping with viewpoint change

We consider real-time visual tracking with targets undergoing viewpoint changes. The problem is evaluated on a new and extensive dataset of vehicles undergoing large viewpoint changes. We propose an evaluation method in which tracking accuracy is measured under real-time computational complexity constraints and find that state-of-the-art agnostic trackers, as well as class detectors, are still struggling with this task. We study tracking schemes fusing real-time agnostic trackers with a non-real-time class detector used for template update, with two dominating update strategies emerging. We rigorously analyze the template update latency and demonstrate that such methods significantly outperform stand-alone trackers and class detectors. Results are demonstrated using two different trackers and a state-of-the-art classifier, and at several operating points of algorithm/hardware computational speed.