Ground2sky label transfer for fine-grained aerial car recognition

Overhead images captured by helicopters, unmanned aerial vehicles and satellites are widely available. Prior aerial target recognition methods mainly deal with generic object categories such as cars, roads, and boats. We go beyond this and aim for fine-grained recognition, e.g., distinguishing between a Toyota and a Honda sedan. This task is so challenging for human annotators that labeling images directly is no longer an option: annotators are often unable to identify the object from such an extreme viewpoint and at such a low resolution. We propose a novel solution to collect fine-grained annotations of aerial images and develop the first ground-to-sky cross-view car dataset with instance-level correspondences. We compare the performance of human experts and deep learning approaches on fine-grained car recognition from aerial imagery. Noting that intraclass variation in aerial images is limited, we further show that with simple data augmentation, a classifier can be trained from fewer instances yet achieves comparable or even significantly better performance than human experts. Our experimental evidence demonstrates that fine-grained object recognition from overhead images is not only feasible but also well suited for deep learning methods. Our dataset is available at: http://ai.bu.edu/Ground2Sky/.