Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions
arxiv(2024)
摘要
Generative search engines have the potential to transform how people seek
information online, but generated responses from existing large language models
(LLMs)-backed generative search engines may not always be accurate.
Nonetheless, retrieval-augmented generation exacerbates safety concerns, since
adversaries may successfully evade the entire system by subtly manipulating the
most vulnerable part of a claim. To this end, we propose evaluating the
robustness of generative search engines in the realistic and high-risk setting,
where adversaries have only black-box system access and seek to deceive the
model into returning incorrect responses. Through a comprehensive human
evaluation of various generative search engines, such as Bing Chat,
PerplexityAI, and YouChat across diverse queries, we demonstrate the
effectiveness of adversarial factual questions in inducing incorrect responses.
Moreover, retrieval-augmented generation exhibits a higher susceptibility to
factual errors compared to LLMs without retrieval. These findings highlight the
potential security risks of these systems and emphasize the need for rigorous
evaluation before deployment.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要