GPS-SSL: Guided Positive Sampling to Inject Prior Into Self-Supervised Learning
CoRR(2024)
摘要
We propose Guided Positive Sampling Self-Supervised Learning (GPS-SSL), a
general method to inject a priori knowledge into Self-Supervised Learning (SSL)
positive samples selection. Current SSL methods leverage Data-Augmentations
(DA) for generating positive samples and incorporate prior knowledge - an
incorrect, or too weak DA will drastically reduce the quality of the learned
representation. GPS-SSL proposes instead to design a metric space where
Euclidean distances become a meaningful proxy for semantic relationship. In
that space, it is now possible to generate positive samples from nearest
neighbor sampling. Any prior knowledge can now be embedded into that metric
space independently from the employed DA. From its simplicity, GPS-SSL is
applicable to any SSL method, e.g. SimCLR or BYOL. A key benefit of GPS-SSL is
in reducing the pressure in tailoring strong DAs. For example GPS-SSL reaches
85.58
therefore move a step forward towards the goal of making SSL less reliant on
DA. We also show that even when using strong DAs, GPS-SSL outperforms the
baselines on under-studied domains. We evaluate GPS-SSL along with multiple
baseline SSL methods on numerous downstream datasets from different domains
when the models use strong or minimal data augmentations. We hope that GPS-SSL
will open new avenues in studying how to inject a priori knowledge into SSL in
a principled manner.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要