ESP-Zero: Unsupervised enhancement of zero-shot classification for Extremely Sparse Point cloud
arxiv(2024)
摘要
In recent years, zero-shot learning has attracted the focus of many
researchers, due to its flexibility and generality. Many approaches have been
proposed to achieve the zero-shot classification of the point clouds for 3D
object understanding, following the schema of CLIP. However, in the real world,
the point clouds could be extremely sparse, dramatically limiting the
effectiveness of the 3D point cloud encoders, and resulting in the misalignment
of point cloud features and text embeddings. To the point cloud encoders to fit
the extremely sparse point clouds without re-running the pre-training procedure
which could be time-consuming and expensive, in this work, we propose an
unsupervised model adaptation approach to enhance the point cloud encoder for
the extremely sparse point clouds. We propose a novel fused-cross attention
layer that expands the pre-trained self-attention layer with additional
learnable tokens and attention blocks, which effectively modifies the point
cloud features while maintaining the alignment between point cloud features and
text embeddings. We also propose a complementary learning-based
self-distillation schema that encourages the modified features to be pulled
apart from the irrelevant text embeddings without overfitting the feature space
to the observed text embeddings. Extensive experiments demonstrate that the
proposed approach effectively increases the zero-shot capability on extremely
sparse point clouds, and overwhelms other state-of-the-art model adaptation
approaches.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要