Axis Tour: Word Tour Determines the Order of Axes in ICA-transformed Embeddings
CoRR(2024)
摘要
Word embedding is one of the most important components in natural language
processing, but interpreting high-dimensional embeddings remains a challenging
problem. To address this problem, Independent Component Analysis (ICA) is
identified as an effective solution. ICA-transformed word embeddings reveal
interpretable semantic axes; however, the order of these axes are arbitrary. In
this study, we focus on this property and propose a novel method, Axis Tour,
which optimizes the order of the axes. Inspired by Word Tour, a one-dimensional
word embedding method, we aim to improve the clarity of the word embedding
space by maximizing the semantic continuity of the axes. Furthermore, we show
through experiments on downstream tasks that Axis Tour constructs better
low-dimensional embeddings compared to both PCA and ICA.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要