How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts
CoRR(2024)
摘要
The remarkable advancements in Multimodal Large Language Models (MLLMs) have
not rendered them immune to challenges, particularly in the context of handling
deceptive information in prompts, thus producing hallucinated responses under
such conditions. To quantitatively assess this vulnerability, we present
MAD-Bench, a carefully curated benchmark that contains 850 test samples divided
into 6 categories, such as non-existent objects, count of objects, spatial
relationship, and visual confusion. We provide a comprehensive analysis of
popular MLLMs, ranging from GPT-4V, Gemini-Pro, to open-sourced models, such as
LLaVA-1.5 and CogVLM. Empirically, we observe significant performance gaps
between GPT-4V and other models; and previous robust instruction-tuned models,
such as LRV-Instruction and LLaVA-RLHF, are not effective on this new
benchmark. While GPT-4V achieves 75.02
any other model in our experiments ranges from 5
remedy that adds an additional paragraph to the deceptive prompts to encourage
models to think twice before answering the question. Surprisingly, this simple
method can even double the accuracy; however, the absolute numbers are still
too low to be satisfactory. We hope MAD-Bench can serve as a valuable benchmark
to stimulate further research to enhance models' resilience against deceptive
prompts.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要