Leveraging vocabulary tree for simultaneous match pair selection and guided feature matching of UAV images

Structure from Motion (SfM) has been a golden standard technique for UAV image orientation. However, the high combinational complexity of match pairs and the high outlier ratio of initial matches become two major issues in SfM-based image orientation. This paper presents an integrated workflow to achieve simultaneously match pair selection and guided feature matching for image orientation. The core idea of the proposed algorithm is to explore the index structure of both inverted and direct indexes in the context of vocabulary tree-based image retrieval. First, the similarity scores between one query image and database images are calculated by using the word-image index structure stored in the inverted index, and match pairs are selected with a distance or ratio threshold-independent strategy. Second, by using a large vocabulary tree for descriptor quantification, guided feature matching is achieved by using the image-word index structure stored in the direct index, which restricts the searching space of nearest descriptors and achieves the direct assignment of candidate matches, instead of the nearest neighbor searching (NNS) method. Finally, integrated with an incremental SfM workflow, the performance of the proposed algorithm has been comprehensively verified through the analysis and comparison by using four UAV datasets that cover varying test sites. The experimental results demonstrate that the proposed method achieves match pair selection with linear time complexity and provides refined matches with speedup ratios ranging from 156 to 228 (2 to 3) compared with the CPU (GPU)-based NNS matching method. In addition, it can achieve competitive precision in both relative bundle adjustment (BA) without ground control points (GCPs) and absolute BA with GCPs.