Hierarchical Quantization For Billion-Scale Similarity Retrieval

Existing billion-scale approximate nearest neighbor (ANN) search methods are usually based on GPU implementations. However, the search accuracy of these methods are limited by their indexing structures and encoding algorithms which are not adaptable to GPUs.We introduce an ANN search method called Vector and Product Quantization Tree (V-PQT) based on GPUs, based on a novel two-level tree indexing structure that increase indexing efficiency and a novel encoding algorithm that improved encoding quality of previous quantization methods by a large margin.Comparing to other state-of-the-art methods based on GPUs, V-PQT system can provide a significantly better recall with the same level of runtime. The experimental result shows that the proposed method has 27% and 20% improved to the conventional methods in the term of search accuracy on two public billion-scale datasets.