Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentation
CoRR(2024)
摘要
The Segment Anything Model (SAM), a profound vision foundation model
pre-trained on a large-scale dataset, breaks the boundaries of general
segmentation and sparks various downstream applications. This paper introduces
Hi-SAM, a unified model leveraging SAM for hierarchical text segmentation.
Hi-SAM excels in text segmentation across four hierarchies, including stroke,
word, text-line, and paragraph, while realizing layout analysis as well.
Specifically, we first turn SAM into a high-quality text stroke segmentation
(TSS) model through a parameter-efficient fine-tuning approach. We use this TSS
model to iteratively generate the text stroke labels in a semi-automatical
manner, unifying labels across the four text hierarchies in the HierText
dataset. Subsequently, with these complete labels, we launch the end-to-end
trainable Hi-SAM based on the TSS architecture with a customized hierarchical
mask decoder. During inference, Hi-SAM offers both automatic mask generation
(AMG) mode and promptable segmentation mode. In terms of the AMG mode, Hi-SAM
segments text stroke foreground masks initially, then samples foreground points
for hierarchical text mask generation and achieves layout analysis in passing.
As for the promptable mode, Hi-SAM provides word, text-line, and paragraph
masks with a single point click. Experimental results show the state-of-the-art
performance of our TSS model: 84.86
TextSeg for text stroke segmentation. Moreover, compared to the previous
specialist for joint hierarchical detection and layout analysis on HierText,
Hi-SAM achieves significant improvements: 4.73
text-line level, 5.49
requiring 20x fewer training epochs. The code is available at
https://github.com/ymy-k/Hi-SAM.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要