Abstract
Existing scene text recognition (STR) methods struggle to recognize challenging texts, especially for artistic and severely distorted characters. The limitation lies in the insufficient exploration of character morphologies, including the monotonousness of widely used synthetic training data and the sensitivity of the model to character morphologies. To address these issues, inspired by the human learning process of viewing and summarizing, we facilitate the contrastive learning-based STR framework in a self-motivated manner by leveraging synthetic and real unlabeled data without any human cost. In the viewing process, to compensate for the simplicity of synthetic data and enrich character morphology diversity, we propose an Online Generation Strategy to generate background-free samples with diverse character styles. By excluding background noise distractions, the model is encouraged to focus on character morphology and generalize the ability to recognize complex samples when trained with only simple synthetic data. To boost the summarizing process, we theoretically demonstrate the derivation error in the previous character contrastive loss, which mistakenly causes the sparsity in the intra-class distribution and exacerbates ambiguity on challenging samples. Therefore, a new Character Unidirectional Alignment Loss is proposed to correct this error and unify the representation of the same characters in all samples by aligning the character features in the student model with the reference features in the teacher model. Extensive experiment results show that our method achieves SOTA performance (94.7\% and 70.9\% average accuracy on common benchmarks and Union14M-Benchmark). Code will be available at this https URL.
Abstract (translated)
现有的场景文本识别（STR）方法在识别具有挑战性的文本方面存在困难，特别是对于艺术性和严重扭曲的字符。其局限性在于对字符形态探索不足，包括广泛使用的合成训练数据单调以及模型对字符形态敏感。为了解决这些问题，受人类学习过程中的观察和总结启发，我们通过利用合成和现实未标记的数据，在无需人工成本的情况下，以自我激励的方式促进对比学习为基础的STR框架的发展。在观察过程中，为了弥补合成数据的简单性并丰富字符形态多样性，我们提出了一种在线生成策略来生成具有多样化字符风格且无背景的样本。通过排除背景噪音干扰，模型被鼓励专注于字符形态，并能够仅使用简单的合成数据进行训练时泛化识别复杂样本的能力。为了增强总结过程，我们在理论上证明了先前字符对比损失中的推导错误，该错误误导致类内分布稀疏并加剧了对具有挑战性样本的模糊性。因此，我们提出了一个新的字符单向对齐损失来纠正这一错误并通过在学生模型中将字符特征与教师模型中的参考特征对齐统一所有样本中相同字符的表现形式。广泛的实验结果表明，我们的方法实现了最先进的性能（在常见基准和Union14M-Benchmark上的平均准确率分别为94.7%和70.9%）。代码将在以下链接提供：此 https URL。
URL
https://arxiv.org/abs/2411.15585
