Countering Bias In Tracking Evaluations

Recent years have witnessed a significant leap in visual object tracking performance mainly due to powerful features, sophisticated learning methods and the introduction of benchmark datasets. Despite this significant improvement, the evaluation of state-of-the-art object trackers still relies on the classical intersection over union (IoU) score. In this work, we argue that the object tracking evaluations based on classical IoU score are sub-optimal. As our first contribution, we theoretically prove that the IoU score is biased in the case of large target objects and favors over-estimated target prediction sizes. As our second contribution, we propose a new score that is unbiased with respect to target prediction size. We systematically evaluate our proposed approach on benchmark tracking data with variations in relative target size. Our empirical results clearly suggest that the proposed score is unbiased in general.