Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal
CVPR 2024(2024)
摘要
Real-world vision tasks frequently suffer from the appearance of unexpected
adverse weather conditions, including rain, haze, snow, and raindrops. In the
last decade, convolutional neural networks and vision transformers have yielded
outstanding results in single-weather video removal. However, due to the
absence of appropriate adaptation, most of them fail to generalize to other
weather conditions. Although ViWS-Net is proposed to remove adverse weather
conditions in videos with a single set of pre-trained weights, it is seriously
blinded by seen weather at train-time and degenerates when coming to unseen
weather during test-time. In this work, we introduce test-time adaptation into
adverse weather removal in videos, and propose the first framework that
integrates test-time adaptation into the iterative diffusion reverse process.
Specifically, we devise a diffusion-based network with a novel temporal noise
model to efficiently explore frame-correlated information in degraded video
clips at training stage. During inference stage, we introduce a proxy task
named Diffusion Tubelet Self-Calibration to learn the primer distribution of
test video stream and optimize the model by approximating the temporal noise
model for online adaptation. Experimental results, on benchmark datasets,
demonstrate that our Test-Time Adaptation method with Diffusion-based
network(Diff-TTA) outperforms state-of-the-art methods in terms of restoring
videos degraded by seen weather conditions. Its generalizable capability is
also validated with unseen weather conditions in both synthesized and
real-world videos.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要