Abstract
Most previous scene text spotting methods rely on high-quality manual annotations to achieve promising performance. To reduce their expensive costs, we study semi-supervised text spotting (SSTS) to exploit useful information from unlabeled images. However, directly applying existing semi-supervised methods of general scenes to SSTS will face new challenges: 1) inconsistent pseudo labels between detection and recognition tasks, and 2) sub-optimal supervisions caused by inconsistency between teacher/student. Thus, we propose a new Semi-supervised framework for End-to-end Text Spotting, namely SemiETS that leverages the complementarity of text detection and recognition. Specifically, it gradually generates reliable hierarchical pseudo labels for each task, thereby reducing noisy labels. Meanwhile, it extracts important information in locations and transcriptions from bidirectional flows to improve consistency. Extensive experiments on three datasets under various settings demonstrate the effectiveness of SemiETS on arbitrary-shaped text. For example, it outperforms previous state-of-the-art SSL methods by a large margin on end-to-end spotting (+8.7%, +5.6%, and +2.6% H-mean under 0.5%, 1%, and 2% labeled data settings on Total-Text, respectively). More importantly, it still improves upon a strongly supervised text spotter trained with plenty of labeled data by 2.0%. Compelling domain adaptation ability shows practical potential. Moreover, our method demonstrates consistent improvement on different text spotters.
Abstract (translated)
大多数以前的场景文本识别方法依赖于高质量的手工标注来实现较好的性能。为了减少昂贵的成本，我们研究了半监督文本定位（SSTS），以利用未标记图像中的有用信息。然而，直接将现有的通用场景半监督方法应用于SSTS会面临新的挑战：1）检测和识别任务之间的伪标签不一致；2）由于教师/学生之间的一致性问题导致的次优监督信号。因此，我们提出了一种新的用于端到端文本定位的半监督框架——SemiETS，该框架利用了文本检测与识别间的互补关系。具体来说，它逐步为每个任务生成可靠的分层伪标签，从而减少噪声标签的数量。同时，它从双向流动中提取位置和转录中的重要信息以提高一致性。在三个不同设置的数据集上的广泛实验表明，SemiETS在任意形状文本的定位上表现出有效性。例如，在Total-Text数据集中，对于端到端检测任务，其分别在0.5%，1% 和2% 标注数据的条件下超过了之前的最先进SSL方法8.7%，5.6%，和2.6%H-means。更重要的是，它仍然比经过大量标注数据训练的强大监督文本定位器提高了2.0% 的性能。强大的领域适应能力显示了其实用潜力。此外，我们的方法在不同的文本定位器上均表现出一致的改进效果。
URL
https://arxiv.org/abs/2504.09966
