The Power of Noise: Toward a Unified Multi-modal Knowledge Graph Representation Framework
arxiv(2024)
摘要
The advancement of Multi-modal Pre-training highlights the necessity for a
robust Multi-Modal Knowledge Graph (MMKG) representation learning framework.
This framework is crucial for integrating structured knowledge into multi-modal
Large Language Models (LLMs) at scale, aiming to alleviate issues like
knowledge misconceptions and multi-modal hallucinations. In this work, to
evaluate models' ability to accurately embed entities within MMKGs, we focus on
two widely researched tasks: Multi-modal Knowledge Graph Completion (MKGC) and
Multi-modal Entity Alignment (MMEA). Building on this foundation, we propose a
novel SNAG method that utilizes a Transformer-based architecture equipped with
modality-level noise masking for the robust integration of multi-modal entity
features in KGs. By incorporating specific training objectives for both MKGC
and MMEA, our approach achieves SOTA performance across a total of ten datasets
(three for MKGC and seven for MEMA), demonstrating its robustness and
versatility. Besides, SNAG can not only function as a standalone model but also
enhance other existing methods, providing stable performance improvements. Our
code and data are available at: https://github.com/zjukg/SNAG.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要