MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo
CoRR(2024)
摘要
Recent advancements in learning-based Multi-View Stereo (MVS) methods have
prominently featured transformer-based models with attention mechanisms.
However, existing approaches have not thoroughly investigated the profound
influence of transformers on different MVS modules, resulting in limited depth
estimation capabilities. In this paper, we introduce MVSFormer++, a method that
prudently maximizes the inherent characteristics of attention to enhance
various components of the MVS pipeline. Formally, our approach involves
infusing cross-view information into the pre-trained DINOv2 model to facilitate
MVS learning. Furthermore, we employ different attention mechanisms for the
feature encoder and cost volume regularization, focusing on feature and spatial
aggregations respectively. Additionally, we uncover that some design details
would substantially impact the performance of transformer modules in MVS,
including normalized 3D positional encoding, adaptive attention scaling, and
the position of layer normalization. Comprehensive experiments on DTU,
Tanks-and-Temples, BlendedMVS, and ETH3D validate the effectiveness of the
proposed method. Notably, MVSFormer++ achieves state-of-the-art performance on
the challenging DTU and Tanks-and-Temples benchmarks.
更多查看译文
关键词
Multi-View Stereo,Transformer,Depth Estimation
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要