Adaptive Self-training Framework for Fine-grained Scene Graph Generation
CoRR(2024)
摘要
Scene graph generation (SGG) models have suffered from inherent problems
regarding the benchmark datasets such as the long-tailed predicate distribution
and missing annotation problems. In this work, we aim to alleviate the
long-tailed problem of SGG by utilizing unannotated triplets. To this end, we
introduce a Self-Training framework for SGG (ST-SGG) that assigns pseudo-labels
for unannotated triplets based on which the SGG models are trained. While there
has been significant progress in self-training for image recognition, designing
a self-training framework for the SGG task is more challenging due to its
inherent nature such as the semantic ambiguity and the long-tailed distribution
of predicate classes. Hence, we propose a novel pseudo-labeling technique for
SGG, called Class-specific Adaptive Thresholding with Momentum (CATM), which is
a model-agnostic framework that can be applied to any existing SGG models.
Furthermore, we devise a graph structure learner (GSL) that is beneficial when
adopting our proposed self-training framework to the state-of-the-art
message-passing neural network (MPNN)-based SGG models. Our extensive
experiments verify the effectiveness of ST-SGG on various SGG models,
particularly in enhancing the performance on fine-grained predicate classes.
更多查看译文
关键词
Scene Graph Generation,Scene Understanding,Imbalanced Classification,Self-training,Long-tailed Problem
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要