Efficient Adapter Finetuning for Tail Languages in Streaming Multilingual ASR
CoRR(2024)
摘要
The end-to-end ASR model is often desired in the streaming multilingual
scenario since it is easier to deploy and can benefit from pre-trained speech
models such as powerful foundation models. Meanwhile, the heterogeneous nature
and imbalanced data abundance of different languages may cause performance
degradation, leading to asynchronous peak performance for different languages
during training, especially on tail ones. Sometimes even the data itself may
become unavailable as a result of the enhanced privacy protection. Existing
work tend to significantly increase the model size or learn language-specific
decoders to accommodate each language separately. In this study, we explore
simple yet effective Language-Dependent Adapter (LDA) finetuning under a
cascaded Conformer transducer framework enhanced by teacher pseudo-labeling for
tail languages in the streaming multilingual ASR. The adapter only accounts for
0.4
model and is the only trainable module during the finetuning process with noisy
student training. The final model merges the adapter parameters from different
checkpoints for different languages. The model performance is validated on a
challenging multilingual dictation dataset, which includes 39 tail languages
across Latin, Greek, Arabic, etc. Our proposed method brings 12.2
rate reduction on average and up to 37.5
show that our parameter-efficient LDA can match the quality of the full model
finetuning, thus greatly alleviating the asynchronous peak performance issue.
更多查看译文
关键词
Streaming Multilingual ASR,Adapter Finetuning
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要