Distilled Datamodel with Reverse Gradient Matching
CVPR 2024(2024)
摘要
The proliferation of large-scale AI models trained on extensive datasets has
revolutionized machine learning. With these models taking on increasingly
central roles in various applications, the need to understand their behavior
and enhance interpretability has become paramount. To investigate the impact of
changes in training data on a pre-trained model, a common approach is
leave-one-out retraining. This entails systematically altering the training
dataset by removing specific samples to observe resulting changes within the
model. However, retraining the model for each altered dataset presents a
significant computational challenge, given the need to perform this operation
for every dataset variation. In this paper, we introduce an efficient framework
for assessing data impact, comprising offline training and online evaluation
stages. During the offline training phase, we approximate the influence of
training data on the target model through a distilled synset, formulated as a
reversed gradient matching problem. For online evaluation, we expedite the
leave-one-out process using the synset, which is then utilized to compute the
attribution matrix based on the evaluation objective. Experimental evaluations,
including training data attribution and assessments of data quality,
demonstrate that our proposed method achieves comparable model behavior
evaluation while significantly speeding up the process compared to the direct
retraining method.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要