A Simple And Effective Unified Encoder For Document-Level Machine Translation

Most of the existing models for document-level machine translation adopt dual-encoder structures. The representation of the source sentences and the document-level contexts' are modeled with two separate encoders. Although these models can make use of the document-level contexts, they do not fully model the interaction between the contexts and the source sentences, and can not directly adapt to the recent pre-training models (e.g., BERT) which encodes multiple sentences with a single encoder. In this work, we propose a simple and effective unified encoder that can outperform the baseline models of dual encoder models in terms of BLEU and METLOR scores. Moreover, the pre-training models can further boost the performance of our proposed model.