Distillation is All You Need for Practically Using Different Pre-trained Recommendation Models
CoRR(2024)
摘要
Pre-trained recommendation models (PRMs) have attracted widespread attention
recently. However, their totally different model structure, huge model size and
computation cost hinder their application in practical recommender systems.
Hence, it is highly essential to explore how to practically utilize PRMs in
real-world recommendations. In this paper, we propose a novel joint knowledge
distillation from different pre-trained recommendation models named PRM-KD for
recommendation, which takes full advantages of diverse PRMs as teacher models
for enhancing student models efficiently. Specifically, PRM-KD jointly distills
diverse informative knowledge from multiple representative PRMs such as
UniSRec, Recformer, and UniM^2Rec. The knowledge from the above PRMs are then
smartly integrated into the student recommendation model considering their
confidence and consistency. We further verify the universality of PRM-KD with
various types of student models, including sequential recommendation, feature
interaction, and graph-based models. Extensive experiments on five real-world
datasets demonstrate the effectiveness and efficacy of PRM-KD, which could be
viewed as an economical shortcut in practically and conveniently making full
use of different PRMs in online systems.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要