Triple-Encoders: Representations That Fire Together, Wire Together
CoRR(2024)
摘要
Search-based dialog models typically re-encode the dialog history at every
turn, incurring high cost. Curved Contrastive Learning, a representation
learning method that encodes relative distances between utterances into the
embedding space via a bi-encoder, has recently shown promising results for
dialog modeling at far superior efficiency. While high efficiency is achieved
through independently encoding utterances, this ignores the importance of
contextualization. To overcome this issue, this study introduces
triple-encoders, which efficiently compute distributed utterance mixtures from
these independently encoded utterances through a novel hebbian inspired
co-occurrence learning objective without using any weights. Empirically, we
find that triple-encoders lead to a substantial improvement over bi-encoders,
and even to better zero-shot generalization than single-vector representation
models without requiring re-encoding. Our code/model is publicly available.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要