Usgg: union message based scene graph generation

Scene graph generation (SGG) is designed to represent images by objects and their relationships. Existing works mainly attempt to strengthen object pair representations for SGG. However, most methods ignore the significant semantic information implied in union regions, which refers to the surrounding area of object pairs. In this paper, we propose a new union message based architecture, named as USGG, to profoundly exploit the relational semantics of unions to facilitate SGG. Concretely, we employ sufficient feature extraction to enhance the features of objects and unions. Next, we devise the Union Embedding Network to model the relational representations through two symmetric encoder-decoder branches. Moreover, the Union Fusion Network is designed to integrate the refined semantics by two-stage feature fusion. Extensive experiments are conducted on Visual Genome dataset, which demonstrates that the proposed approach achieves competitive performance against state-of-the-art methods on Recall, mean Recall and Zero Shot Recall metrics.