Distilling Large Language Models for Text-Attributed Graph Learning
CoRR(2024)
摘要
Text-Attributed Graphs (TAGs) are graphs of connected textual documents.
Graph models can efficiently learn TAGs, but their training heavily relies on
human-annotated labels, which are scarce or even unavailable in many
applications. Large language models (LLMs) have recently demonstrated
remarkable capabilities in few-shot and zero-shot TAG learning, but they suffer
from scalability, cost, and privacy issues. Therefore, in this work, we focus
on synergizing LLMs and graph models with their complementary strengths by
distilling the power of LLMs to a local graph model on TAG learning. To address
the inherent gaps between LLMs (generative models for texts) and graph models
(discriminative models for graphs), we propose first to let LLMs teach an
interpreter with rich textual rationale and then let a student model mimic the
interpreter's reasoning without LLMs' textual rationale. Extensive experiments
validate the efficacy of our proposed framework.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要