Vaccine: Perturbation-aware Alignment for Large Language Model
CoRR(2024)
摘要
The new paradigm of finetuning-as-a-service introduces a new attack surface
for Large Language Models (LLMs): a few harmful data uploaded by users can
easily trick the finetuning to produce an alignment-broken model. We conduct an
empirical analysis and uncover a harmful embedding drift phenomenon,
showing a probable cause of the alignment-broken effect. Inspired by our
findings, we propose Vaccine, a perturbation-aware alignment technique to
mitigate the security risk of users finetuning. The core idea of Vaccine is to
produce invariant hidden embeddings by progressively adding crafted
perturbation to them in the alignment phase. This enables the embeddings to
withstand harmful perturbation from un-sanitized user data in the finetuning
phase. Our results on open source mainstream LLMs (e.g., Llama2, Opt, Vicuna)
demonstrate that Vaccine can boost the robustness of alignment against harmful
prompts induced embedding drift while reserving reasoning ability towards
benign prompts. Our code is available at
.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要