Efficient Analogy Completion with Word Embedding Clusters.

Word embeddings have attracted much attention in recent years and have been heavily applied to many tasks in information retrieval, natural language processing and knowledge base construction. One of the most well noted aspects of word embeddings is their ability to capture relations between terms via simple vector offsets. This ability is often examined via the use of proportional analogy completion tasks. This task requires that the correct single term be returned by a system when prompted with the three other terms of a proportional analogy. This task usually involves a scan of all stored word embeddings which may be a relatively expensive operation when used as part of a larger system. In some preliminary experiments we show that it is quite easy to achieve significant speedups for analogy completion, with little loss of accuracy, via the use of a simple clustering solution of word vectors. Cluster trees have long been used to improve the efficiency of nearest neighbour retrieval but analogy completion has some additional structure due to the use of analogy completion formulas. We evaluate several configurations of word embedding clusters. Due to its relationship to tasks such as link prediction and knowledge base completion we believe that these results may be of interest to a wider group of people.