Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal
arxiv(2024)
摘要
Large language models (LLMs) suffer from catastrophic forgetting during
continual learning. Conventional rehearsal-based methods rely on previous
training data to retain the model's ability, which may not be feasible in
real-world applications. When conducting continual learning based on a
publicly-released LLM checkpoint, the availability of the original training
data may be non-existent. To address this challenge, we propose a framework
called Self-Synthesized Rehearsal (SSR) that uses the LLM to generate synthetic
instances for rehearsal. Concretely, we first employ the base LLM for
in-context learning to generate synthetic instances. Subsequently, we utilize
the latest LLM to refine the instance outputs based on the synthetic inputs,
preserving its acquired ability. Finally, we select diverse high-quality
synthetic instances for rehearsal in future stages. Experimental results
demonstrate that SSR achieves superior or comparable performance compared to
conventional rehearsal-based approaches while being more data-efficient.
Besides, SSR effectively preserves the generalization capabilities of LLMs in
general domains.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要