A Gaze-grounded Visual Question Answering Dataset for Clarifying Ambiguous Japanese Questions
arxiv(2024)
摘要
Situated conversations, which refer to visual information as visual question
answering (VQA), often contain ambiguities caused by reliance on directive
information. This problem is exacerbated because some languages, such as
Japanese, often omit subjective or objective terms. Such ambiguities in
questions are often clarified by the contexts in conversational situations,
such as joint attention with a user or user gaze information. In this study, we
propose the Gaze-grounded VQA dataset (GazeVQA) that clarifies ambiguous
questions using gaze information by focusing on a clarification process
complemented by gaze information. We also propose a method that utilizes gaze
target estimation results to improve the accuracy of GazeVQA tasks. Our
experimental results showed that the proposed method improved the performance
in some cases of a VQA system on GazeVQA and identified some typical problems
of GazeVQA tasks that need to be improved.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要