Abstract
Recent advancements in scene text spotting have focused on end-to-end methodologies that heavily rely on precise location annotations, which are often costly and labor-intensive to procure. In this study, we introduce an innovative approach that leverages only transcription annotations for training text spotting models, substantially reducing the dependency on elaborate annotation processes. Our methodology employs a query-based paradigm that facilitates the learning of implicit location features through the interaction between text queries and image embeddings. These features are later refined during the text recognition phase using an attention activation map. Addressing the challenges associated with training a weakly-supervised model from scratch, we implement a circular curriculum learning strategy to enhance model convergence. Additionally, we introduce a coarse-to-fine cross-attention localization mechanism for more accurate text instance localization. Notably, our framework supports audio-based annotation, which significantly diminishes annotation time and provides an inclusive alternative for individuals with disabilities. Our approach achieves competitive performance against existing benchmarks, demonstrating that high accuracy in text spotting can be attained without extensive location annotations.
Abstract (translated)
近期的场景文本识别研究主要集中在端到端的方法上，这些方法依赖于精确的位置标注，而获取这种标注通常既昂贵又费时。在本研究中，我们提出了一种创新的方法，仅利用转录标注来训练文本识别模型，从而大大减少了对复杂标注过程的依赖。我们的方法采用基于查询的范式，在该框架下，通过文本查询与图像嵌入之间的交互学习隐含位置特征，并在后续的文字识别阶段使用注意力激活图进一步细化这些特征。 为了解决从零开始训练弱监督模型时面临的挑战，我们实施了循环课程学习策略来提高模型收敛性。此外，为了实现更准确的文本实例定位，我们引入了一种由粗到细的跨注意力定位机制。值得一提的是，我们的框架还支持基于音频的标注方法，这显著减少了标注时间，并为残疾人士提供了包容性的替代方案。 在现有的基准测试中，我们的方法达到了与现有技术相当的表现水平，证明了即使没有大量的位置标注，也能够实现高水平的文字识别准确率。
URL
https://arxiv.org/abs/2412.19504
