Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learning
CoRR(2024)
摘要
In this study, we present aLLM4TS, an innovative framework that adapts Large
Language Models (LLMs) for time-series representation learning. Central to our
approach is that we reconceive time-series forecasting as a self-supervised,
multi-patch prediction task, which, compared to traditional
mask-and-reconstruction methods, captures temporal dynamics in patch
representations more effectively. Our strategy encompasses two-stage training:
(i). a causal continual pre-training phase on various time-series datasets,
anchored on next patch prediction, effectively syncing LLM capabilities with
the intricacies of time-series data; (ii). fine-tuning for multi-patch
prediction in the targeted time-series context. A distinctive element of our
framework is the patch-wise decoding layer, which departs from previous methods
reliant on sequence-level decoding. Such a design directly transposes
individual patches into temporal sequences, thereby significantly bolstering
the model's proficiency in mastering temporal patch-based representations.
aLLM4TS demonstrates superior performance in several downstream tasks, proving
its effectiveness in deriving temporal representations with enhanced
transferability and marking a pivotal advancement in the adaptation of LLMs for
time-series analysis.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要