Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentations
CoRR(2024)
摘要
Models trained on a labeled source domain (e.g., labeled images from wildlife
camera traps) often generalize poorly when deployed on an out-of-distribution
(OOD) target domain (e.g., images from new camera trap locations). In the
domain adaptation setting where unlabeled target data is available,
self-supervised pretraining (e.g., masked autoencoding or contrastive learning)
is a promising method to mitigate this performance drop. Pretraining improves
OOD error when the generic data augmentations used (e.g., masking or cropping)
connect the source and target domains, which may be far apart in the input
space. In this paper, we show on real-world tasks that standard fine-tuning
after pretraining does not consistently improve OOD error over simply training
from scratch on labeled source data. To better leverage pretraining for
distribution shifts, we propose Connect Later: after pretraining with generic
augmentations, fine-tune with targeted augmentations designed with knowledge of
the distribution shift. Pretraining learns good representations within the
source and target domains, while targeted augmentations connect the domains
better during fine-tuning. Connect Later improves average OOD error over
standard fine-tuning and supervised learning with targeted augmentations on 4
real-world datasets: Connect Later achieves the state-of-the-art on
astronomical time-series classification (AstroClassification) by 2.5
species identification (iWildCam-WILDS) with ResNet-50 by 0.9
identification (Camelyon17-WILDS) with DenseNet121 by 1.1
performance on a new dataset for astronomical time-series redshift prediction
(Redshifts) by 0.03 RMSE (11
https://github.com/helenqu/connect-later.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要