Customizing Segmentation Foundation Model via Prompt Learning for Instance Segmentation
arxiv(2024)
摘要
Recently, foundation models trained on massive datasets to adapt to a wide
range of domains have attracted considerable attention and are actively being
explored within the computer vision community. Among these, the Segment
Anything Model (SAM) stands out for its remarkable progress in generalizability
and flexibility for image segmentation tasks, achieved through prompt-based
object mask generation. However, despite its strength, SAM faces two key
limitations when applied to customized instance segmentation that segments
specific objects or those in unique environments not typically present in the
training data: 1) the ambiguity inherent in input prompts and 2) the necessity
for extensive additional training to achieve optimal segmentation. To address
these challenges, we propose a novel method, customized instance segmentation
via prompt learning tailored to SAM. Our method involves a prompt learning
module (PLM), which adjusts input prompts into the embedding space to better
align with user intentions, thereby enabling more efficient training.
Furthermore, we introduce a point matching module (PMM) to enhance the feature
representation for finer segmentation by ensuring detailed alignment with
ground truth boundaries. Experimental results on various customized instance
segmentation scenarios demonstrate the effectiveness of the proposed method.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要