InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration
CoRR(2024)
摘要
Though Large Language Models (LLMs) have shown remarkable open-generation
capabilities across diverse domains, they struggle with knowledge-intensive
tasks. To alleviate this issue, knowledge integration methods have been
proposed to enhance LLMs with domain-specific knowledge graphs using external
modules. However, they suffer from data inefficiency as they require both known
and unknown knowledge for fine-tuning. Thus, we study a novel problem of
integrating unknown knowledge into LLMs efficiently without unnecessary overlap
of known knowledge. Injecting new knowledge poses the risk of forgetting
previously acquired knowledge. To tackle this, we propose a novel
Infuser-Guided Knowledge Integration (InfuserKI) framework that utilizes
transformer internal states to determine whether to enhance the original LLM
output with additional information, thereby effectively mitigating knowledge
forgetting. Evaluations on the UMLS-2.5k and MetaQA domain knowledge graphs
demonstrate that InfuserKI can effectively acquire new knowledge and outperform
state-of-the-art baselines by 9
forgetting.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要