Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science
CoRR(2024)
摘要
Intelligent agents powered by large language models (LLMs) have demonstrated
substantial promise in autonomously conducting experiments and facilitating
scientific discoveries across various disciplines. While their capabilities are
promising, they also introduce novel vulnerabilities that demand careful
consideration for safety. However, there exists a notable gap in the
literature, as there has been no comprehensive exploration of these
vulnerabilities. This position paper fills this gap by conducting a thorough
examination of vulnerabilities in LLM-based agents within scientific domains,
shedding light on potential risks associated with their misuse and emphasizing
the need for safety measures. We begin by providing a comprehensive overview of
the potential risks inherent to scientific LLM agents, taking into account user
intent, the specific scientific domain, and their potential impact on the
external environment. Then, we delve into the origins of these vulnerabilities
and provide a scoping review of the limited existing works. Based on our
analysis, we propose a triadic framework involving human regulation, agent
alignment, and an understanding of environmental feedback (agent regulation) to
mitigate these identified risks. Furthermore, we highlight the limitations and
challenges associated with safeguarding scientific agents and advocate for the
development of improved models, robust benchmarks, and comprehensive
regulations to address these issues effectively.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要