NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis
conf_acl(2023)
摘要
This paper describes our system developed for the SemEval-2023 Task 12
"Sentiment Analysis for Low-resource African Languages using Twitter Dataset".
Sentiment analysis is one of the most widely studied applications in natural
language processing. However, most prior work still focuses on a small number
of high-resource languages. Building reliable sentiment analysis systems for
low-resource languages remains challenging, due to the limited training data in
this task. In this work, we propose to leverage language-adaptive and
task-adaptive pretraining on African texts and study transfer learning with
source language selection on top of an African language-centric pretrained
language model. Our key findings are: (1) Adapting the pretrained model to the
target language and task using a small yet relevant corpus improves performance
remarkably by more than 10 F1 score points. (2) Selecting source languages with
positive transfer gains during training can avoid harmful interference from
dissimilar languages, leading to better results in multilingual and
cross-lingual settings. In the shared task, our system wins 8 out of 15 tracks
and, in particular, performs best in the multilingual evaluation.
更多查看译文
关键词
multilingual sentiment analysis,adaptive pretraining,source language,low-resource
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要