Abstract
The recent emergence of the Segment Anything Model (SAM) enables various domain-specific segmentation tasks to be tackled cost-effectively by using bounding boxes as prompts. However, in scene text segmentation, SAM can not achieve desirable performance. The word-level bounding box as prompts is too coarse for characters, while the character-level bounding box as prompts suffers from over-segmentation and under-segmentation issues. In this paper, we propose an automatic annotation pipeline named Char-SAM, that turns SAM into a low-cost segmentation annotator with a Character-level visual prompt. Specifically, leveraging some existing text detection datasets with word-level bounding box annotations, we first generate finer-grained character-level bounding box prompts using the Character Bounding-box Refinement CBR module. Next, we employ glyph information corresponding to text character categories as a new prompt in the Character Glyph Refinement (CGR) module to guide SAM in producing more accurate segmentation masks, addressing issues of over-segmentation and under-segmentation. These modules fully utilize the bbox-to-mask capability of SAM to generate high-quality text segmentation annotations automatically. Extensive experiments on TextSeg validate the effectiveness of Char-SAM. Its training-free nature also enables the generation of high-quality scene text segmentation datasets from real-world datasets like COCO-Text and MLT17.
Abstract (translated)
最近推出的Segment Anything Model (SAM) 使得可以通过使用边界框作为提示来低成本地处理各种特定领域的分割任务。然而，在场景文本分割中，SAM 的表现并不理想。以单词级的边界框作为提示对于字符来说过于粗糙，而以字符级别的边界框作为提示则会导致过度分割和欠分割的问题。为此，我们提出了一种自动注释流水线Char-SAM，它将SAM转变为一种低成本的文字分割标注工具，使用的是基于字符级别的视觉提示。具体而言，利用现有的带有单词级边界框注释的文本检测数据集，我们首先通过Character Bounding-box Refinement (CBR) 模块生成更精细的字符级别边界框提示。接下来，我们在Character Glyph Refinement (CGR) 模块中采用与文字字符类别对应的字形信息作为新的提示，以此来引导SAM产生更加准确的分割掩码，从而解决过度分割和欠分割的问题。这些模块充分利用了SAM将bbox转换为mask的能力，能够自动生成高质量的文字分割标注。在TextSeg上的广泛实验验证了Char-SAM的有效性。其无需训练的特点还使得可以从像COCO-Text 和 MLT17这样的真实世界数据集中生成高质量的场景文本分割数据集。
URL
https://arxiv.org/abs/2412.19917
