The Philosopher's Stone: Trojaning Plugins of Large Language Models
arxiv(2023)
Abstract
Open-source Large Language Models (LLMs) have recently gained popularity
because of their comparable performance to proprietary LLMs. To efficiently
fulfill domain-specialized tasks, open-source LLMs can be refined, without
expensive accelerators, using low-rank adapters. However, it is still unknown
whether low-rank adapters can be exploited to control LLMs. To address this
gap, we demonstrate that an infected adapter can induce, on specific triggers,
an LLM to output content defined by an adversary and to even maliciously use
tools. To train a Trojan adapter, we propose two novel attacks, POLISHED and
FUSION, that improve over prior approaches. POLISHED uses LLM-enhanced
paraphrasing to polish benchmark poisoned datasets. In contrast, in the absence
of a dataset, FUSION leverages an over-poisoning procedure to transform a
benign adaptor. In our experiments, we first conduct two case studies to
demonstrate that a compromised LLM agent can execute malware to control system
(e.g., LLM-driven robot) or launch a spear-phishing attack. Then, in terms of
targeted misinformation, we show that our attacks provide higher attack
effectiveness than the baseline and, for the purpose of attracting downloads,
preserve or improve the adapter's utility. Finally, we design and evaluate
three potential defenses, yet none proved entirely effective in safeguarding
against our attacks.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined