ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation
arxiv(2023)
摘要
Since real-world machine systems are running in non-stationary environments,
Continual Test-Time Adaptation (CTTA) task is proposed to adapt the pre-trained
model to continually changing target domains. Recently, existing methods mainly
focus on model-based adaptation, which aims to leverage a self-training manner
to extract the target domain knowledge. However, pseudo labels can be noisy and
the updated model parameters are unreliable under dynamic data distributions,
leading to error accumulation and catastrophic forgetting in the continual
adaptation process. To tackle these challenges and maintain the model
plasticity, we design a Visual Domain Adapter (ViDA) for CTTA, explicitly
handling both domain-specific and domain-shared knowledge. Specifically, we
first comprehensively explore the different domain representations of the
adapters with trainable high-rank or low-rank embedding spaces. Then we inject
ViDAs into the pre-trained model, which leverages high-rank and low-rank
features to adapt the current domain distribution and maintain the continual
domain-shared knowledge, respectively. To exploit the low-rank and high-rank
ViDAs more effectively, we further propose a Homeostatic Knowledge Allotment
(HKA) strategy, which adaptively combines different knowledge from each ViDA.
Extensive experiments conducted on four widely used benchmarks demonstrate that
our proposed method achieves state-of-the-art performance in both
classification and segmentation CTTA tasks. Note that, our method can be
regarded as a novel transfer paradigm for large-scale models, delivering
promising results in adaptation to continually changing distributions. Project
page: https://sites.google.com/view/iclr2024-vida/home.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要