PartSTAD: 2D-to-3D Part Segmentation Task Adaptation
CoRR(2024)
摘要
We introduce PartSTAD, a method designed for the task adaptation of 2D-to-3D
segmentation lifting. Recent studies have highlighted the advantages of
utilizing 2D segmentation models to achieve high-quality 3D segmentation
through few-shot adaptation. However, previous approaches have focused on
adapting 2D segmentation models for domain shift to rendered images and
synthetic text descriptions, rather than optimizing the model specifically for
3D segmentation. Our proposed task adaptation method finetunes a 2D bounding
box prediction model with an objective function for 3D segmentation. We
introduce weights for 2D bounding boxes for adaptive merging and learn the
weights using a small additional neural network. Additionally, we incorporate
SAM, a foreground segmentation model on a bounding box, to improve the
boundaries of 2D segments and consequently those of 3D segmentation. Our
experiments on the PartNet-Mobility dataset show significant improvements with
our task adaptation approach, achieving a 7.0
improvement in mAP_50 for semantic and instance segmentation compared to the
SotA few-shot 3D segmentation model.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要