REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning
arxiv(2023)
摘要
The success of self-supervised contrastive learning hinges on identifying
positive data pairs, such that when they are pushed together in embedding
space, the space encodes useful information for subsequent downstream tasks.
Constructing positive pairs is non-trivial as the pairing must be similar
enough to reflect a shared semantic meaning, but different enough to capture
within-class variation. Classical approaches in vision use augmentations to
exploit well-established invariances to construct positive pairs, but
invariances in the time-series domain are much less obvious. In our work, we
propose a novel method of using a learned measure for identifying positive
pairs. Our Retrieval-Based Reconstruction (REBAR) measure measures the
similarity between two sequences as the reconstruction error that results from
reconstructing one sequence with retrieved information from the other. Then, if
the two sequences have high REBAR similarity, we label them as a positive pair.
Through validation experiments, we show that the REBAR error is a predictor of
mutual class membership. Once integrated into a contrastive learning framework,
our REBAR method learns an embedding that achieves state-of-the-art performance
on downstream tasks across various modalities.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要