Abstract
Scene text spotting aims to detect and recognize text in real-world images, where instances are often short, fragmented, or visually ambiguous. Existing methods primarily rely on visual cues and implicitly capture local character dependencies, but they overlook the benefits of external linguistic knowledge. Prior attempts to integrate language models either adapt language modeling objectives without external knowledge or apply pretrained models that are misaligned with the word-level granularity of scene text. We propose TiCLS, an end-to-end text spotter that explicitly incorporates external linguistic knowledge from a character-level pretrained language model. TiCLS introduces a linguistic decoder that fuses visual and linguistic features, yet can be initialized by a pretrained language model, enabling robust recognition of ambiguous or fragmented text. Experiments on ICDAR 2015 and Total-Text demonstrate that TiCLS achieves state-of-the-art performance, validating the effectiveness of PLM-guided linguistic integration for scene text spotting.
Abstract (translated)
场景文本识别的目标是在现实世界的图像中检测和识别文字，这些实例往往很短、片段化或视觉上模糊。现有方法主要依赖于视觉线索，并且隐式地捕捉局部字符间的依赖关系，但它们忽略了外部语言知识的好处。以前尝试整合语言模型的方法要么在不使用外部知识的情况下适应语言建模目标，要么应用与场景文本的词级粒度不匹配的预训练模型。 我们提出了TiCLS，这是一种端到端的文字识别器，它明确地从字符级别的预训练语言模型中引入了外部语言知识。TiCLS 引入了一个语言解码器，该解码器融合视觉和语言特征，并且可以通过预训练的语言模型进行初始化，从而能够更稳健地识别模糊或片段化的文本。 在ICDAR 2015和Total-Text数据集上的实验表明，TiCLS 达到了最先进的性能水平，这验证了PLM（预训练语言模型）引导下的语言整合对于场景文本识别的有效性。
URL
https://arxiv.org/abs/2602.04030
