SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detection
CVPR 2024(2024)
摘要
Open-vocabulary object detection (OvOD) has transformed detection into a
language-guided task, empowering users to freely define their class
vocabularies of interest during inference. However, our initial investigation
indicates that existing OvOD detectors exhibit significant variability when
dealing with vocabularies across various semantic granularities, posing a
concern for real-world deployment. To this end, we introduce Semantic Hierarchy
Nexus (SHiNe), a novel classifier that uses semantic knowledge from class
hierarchies. It runs offline in three steps: i) it retrieves relevant
super-/sub-categories from a hierarchy for each target class; ii) it integrates
these categories into hierarchy-aware sentences; iii) it fuses these sentence
embeddings to generate the nexus classifier vector. Our evaluation on various
detection benchmarks demonstrates that SHiNe enhances robustness across diverse
vocabulary granularities, achieving up to +31.9
hierarchies, while retaining improvements using hierarchies generated by large
language models. Moreover, when applied to open-vocabulary classification on
ImageNet-1k, SHiNe improves the CLIP zero-shot baseline by +2.8
SHiNe is training-free and can be seamlessly integrated with any off-the-shelf
OvOD detector, without incurring additional computational overhead during
inference. The code is open source.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要