InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training

Full Text

In this work, we formulate cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts. The unified view helps us to better understand the existing methods for learning cross-lingual representations. More importantly, the information-theoretic framework inspires us to propose a...More