Analysis of Multi-Source Language Training in Cross-Lingual Transfer
CoRR(2024)
摘要
The successful adaptation of multilingual language models (LMs) to a specific
language-task pair critically depends on the availability of data tailored for
that condition. While cross-lingual transfer (XLT) methods have contributed to
addressing this data scarcity problem, there still exists ongoing debate about
the mechanisms behind their effectiveness. In this work, we focus on one of
promising assumptions about inner workings of XLT, that it encourages
multilingual LMs to place greater emphasis on language-agnostic or
task-specific features. We test this hypothesis by examining how the patterns
of XLT change with a varying number of source languages involved in the
process. Our experimental findings show that the use of multiple source
languages in XLT-a technique we term Multi-Source Language Training
(MSLT)-leads to increased mingling of embedding spaces for different languages,
supporting the claim that XLT benefits from making use of language-independent
information. On the other hand, we discover that using an arbitrary combination
of source languages does not always guarantee better performance. We suggest
simple heuristics for identifying effective language combinations for MSLT and
empirically prove its effectiveness.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要