Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shift
CoRR(2024)
摘要
Estimating test accuracy without access to the ground-truth test labels under
varying test environments is a challenging, yet extremely important problem in
the safe deployment of machine learning algorithms. Existing works rely on the
information from either the outputs or the extracted features of neural
networks to formulate an estimation score correlating with the ground-truth
test accuracy. In this paper, we investigate–both empirically and
theoretically–how the information provided by the gradients can be predictive
of the ground-truth test accuracy even under a distribution shift.
Specifically, we use the norm of classification-layer gradients, backpropagated
from the cross-entropy loss after only one gradient step over test data. Our
key idea is that the model should be adjusted with a higher magnitude of
gradients when it does not generalize to the test dataset with a distribution
shift. We provide theoretical insights highlighting the main ingredients of
such an approach ensuring its empirical success. Extensive experiments
conducted on diverse distribution shifts and model structures demonstrate that
our method significantly outperforms state-of-the-art algorithms.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要