InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training

Zewen Chi
Zewen Chi
Nan Yang
Nan Yang
Saksham Singhal
Saksham Singhal
Wenhui Wang
Wenhui Wang
Xia Song
Xia Song
Cited by: 0|Bibtex|Views72|Links

Abstract:

In this work, we formulate cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts. The unified view helps us to better understand the existing methods for learning cross-lingual representations. More importantly, the information-theoretic framework inspires us to propose a...More

Code:

Data:

Your rating :
0

 

Tags
Comments