United We Pretrain, Divided We Fail! Representation Learning for Time Series by Pretraining on 75 Datasets at Once
CoRR(2024)
摘要
In natural language processing and vision, pretraining is utilized to learn
effective representations. Unfortunately, the success of pretraining does not
easily carry over to time series due to potential mismatch between sources and
target. Actually, common belief is that multi-dataset pretraining does not work
for time series! Au contraire, we introduce a new self-supervised contrastive
pretraining approach to learn one encoding from many unlabeled and diverse time
series datasets, so that the single learned representation can then be reused
in several target domains for, say, classification. Specifically, we propose
the XD-MixUp interpolation method and the Soft Interpolation Contextual
Contrasting (SICC) loss. Empirically, this outperforms both supervised training
and other self-supervised pretraining methods when finetuning on low-data
regimes. This disproves the common belief: We can actually learn from multiple
time series datasets, even from 75 at once.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要