Concept-Best-Matching: Evaluating Compositionality in Emergent Communication
arxiv(2024)
摘要
Artificial agents that learn to communicate in order to accomplish a given
task acquire communication protocols that are typically opaque to a human. A
large body of work has attempted to evaluate the emergent communication via
various evaluation measures, with compositionality featuring as a
prominent desired trait. However, current evaluation procedures do not directly
expose the compositionality of the emergent communication. We propose a
procedure to assess the compositionality of emergent communication by finding
the best-match between emerged words and natural language concepts. The
best-match algorithm provides both a global score and a translation-map from
emergent words to natural language concepts. To the best of our knowledge, it
is the first time that such direct and interpretable mapping between emergent
words and human concepts is provided.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要