DEFT: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selection
arxiv(2023)
摘要
Recent advances have led to the availability of many pre-trained language
models (PLMs); however, a question that remains is how much data is truly
needed to fine-tune PLMs for downstream tasks? In this work, we introduce
DEFT-UCS, a data-efficient fine-tuning framework that leverages unsupervised
core-set selection to identify a smaller, representative dataset that reduces
the amount of data needed to fine-tune PLMs for downstream tasks. We examine
the efficacy of DEFT-UCS in the context of text-editing LMs, and compare to the
state-of-the art text-editing model, CoEDIT. Our results demonstrate that
DEFT-UCS models are just as accurate as CoEDIT, across eight different datasets
consisting of six different editing tasks, while finetuned on 70
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要