Task-Agnostic Low-Rank Adapters for Unseen English Dialects.
CoRR(2023)
摘要
Large Language Models (LLMs) are trained on corpora disproportionally
weighted in favor of Standard American English. As a result, speakers of other
dialects experience significantly more failures when interacting with these
technologies. In practice, these speakers often accommodate their speech to be
better understood. Our work shares the belief that language technologies should
be designed to accommodate the diversity in English dialects and not the other
way around. However, prior works on dialect struggle with generalizing to
evolving and emerging dialects in a scalable manner. To fill this gap, our
method, HyperLoRA, leverages expert linguistic knowledge to enable
resource-efficient adaptation via hypernetworks. By disentangling
dialect-specific and cross-dialectal information, HyperLoRA improves
generalization to unseen dialects in a task-agnostic fashion. Not only is
HyperLoRA more scalable in the number of parameters, but it also achieves the
best or most competitive performance across 5 dialects in a zero-shot setting.
In this way, our approach facilitates access to language technology for
billions of English dialect speakers who are traditionally underrepresented.
更多查看译文
关键词
english,task-agnostic,low-rank
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要