Alignment-Aware Word Distance.

Recent approaches [6, 20] formulated the task of unsupervised Semantic Textual Similarity (STS) as Earth Mover’s Distance problem, demonstrating superior interpretability and competitive performance. The main idea behind is using various word distances (or word dissimilarity) as word transportation cost, and then measure text dissimilarity by optimizing the accumulative cost of transporting all words of texts. However, these approaches use static word distance without considering the context of text pairs. Intuitively, the distance of two words tends to contribute more to text dissimilarity if they are well-aligned between texts. Inspired by this observation, we propose Alignment-aware Word Distance (AWD), which leverages prior word alignment information of sentence pairs to refine word transportation cost. Specifically, we design two simple and effective mechanisms to capture prior alignment knowledge via exploiting word position and syntactic dependency, respectively. By incorporating AWD, our method remarkably outperforms current state-of-the-art models on STS tasks.