Can Vision-Language Models be a Good Guesser? Exploring VLMs for Times and Location Reasoning
2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)(2023)
摘要
Vision-Language Models (VLMs) are expected to be capable of reasoning with
commonsense knowledge as human beings. One example is that humans can reason
where and when an image is taken based on their knowledge. This makes us wonder
if, based on visual cues, Vision-Language Models that are pre-trained with
large-scale image-text resources can achieve and even outperform human's
capability in reasoning times and location. To address this question, we
propose a two-stage and probing task,
applied to discriminative and generative VLMs to uncover whether VLMs can
recognize times and location-relevant features and further reason about it. To
facilitate the investigation, we introduce WikiTiLo, a well-curated image
dataset compromising images with rich socio-cultural cues. In the extensive
experimental studies, we find that although VLMs can effectively retain
relevant features in visual encoders, they still fail to make perfect
reasoning. We will release our dataset and codes to facilitate future studies.
更多查看译文
关键词
Algorithms,Image recognition and understanding,Algorithms,Datasets and evaluations,Algorithms,Vision + language and/or other modalities
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要