Quantity Matters: Towards Assessing and Mitigating Number Hallucination in Large Vision-Language Models
arxiv(2024)
摘要
Large-scale vision-language models have demonstrated impressive skill in
handling tasks that involve both areas. Nevertheless, these models frequently
experience significant issues with generating inaccurate information, which is
hallucination. In this study, we concentrate on a specific type of
hallucination-number hallucination, referring to models incorrectly identifying
the number of certain objects in pictures. We perform quantitative evaluations
regarding number hallucination, showing it to be critical in major open-source
large vision-language models. Furthermore, we utilizes two related tasks to
conduct an in-depth analysis of number hallucination, revealing the severe
inner and outer inconsistency among all tasks. Based on this examination, we
devise a training approach aimed at improving consistency to reduce number
hallucinations, which leads to an 8
finetuning methods. Our code and dataset will be released to the community.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要