Reliable Spatial-Temporal Voxels For Multi-Modal Test-Time Adaptation
arxiv(2024)
摘要
Multi-modal test-time adaptation (MM-TTA) is proposed to adapt models to an
unlabeled target domain by leveraging the complementary multi-modal inputs in
an online manner. Previous MM-TTA methods rely on predictions of cross-modal
information in each input frame, while they ignore the fact that predictions of
geometric neighborhoods within consecutive frames are highly correlated,
leading to unstable predictions across time. To fulfill this gap, we propose
ReLiable Spatial-temporal Voxels (Latte), an MM-TTA method that leverages
reliable cross-modal spatial-temporal correspondences for multi-modal 3D
segmentation. Motivated by the fact that reliable predictions should be
consistent with their spatial-temporal correspondences, Latte aggregates
consecutive frames in a slide window manner and constructs ST voxel to capture
temporally local prediction consistency for each modality. After filtering out
ST voxels with high ST entropy, Latte conducts cross-modal learning for each
point and pixel by attending to those with reliable and consistent predictions
among both spatial and temporal neighborhoods. Experimental results show that
Latte achieves state-of-the-art performance on three different MM-TTA
benchmarks compared to previous MM-TTA or TTA methods.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要