Abstract
Scaling architectures have been proven effective for improving Scene Text Recognition (STR), but the individual contribution of vision encoder and text decoder scaling remain under-explored. In this work, we present an in-depth empirical analysis and demonstrate that, contrary to previous observations, scaling the decoder yields significant performance gains, always exceeding those achieved by encoder scaling alone. We also identify label noise as a key challenge in STR, particularly in real-world data, which can limit the effectiveness of STR models. To address this, we propose Cloze Self-Distillation (CSD), a method that mitigates label noise by distilling a student model from context-aware soft predictions and pseudolabels generated by a teacher model. Additionally, we enhance the decoder architecture by introducing differential cross-attention for STR. Our methodology achieves state-of-the-art performance on 10 out of 11 benchmarks using only real data, while significantly reducing the parameter size and computational costs.
Abstract (translated)
扩展架构已被证明在提高场景文本识别（STR）方面是有效的，但视觉编码器和文本解码器的个体贡献仍然未被充分探索。在这项工作中，我们进行了深入的经验分析，并展示了与之前的观察相反，扩大解码器规模能够带来显著的性能提升，这一效果总是超过仅靠编码器扩展所实现的效果。此外，我们还识别出标签噪声作为STR的一个关键挑战，在现实世界的数据中尤其如此，这可能会限制STR模型的有效性。为解决这个问题，我们提出了Cloze自我蒸馏（CSD）方法，该方法通过将学生模型从教师模型生成的上下文感知软预测和伪标签中进行知识蒸馏来减少标签噪声。同时，为了增强解码器架构，我们在STR中引入了差异交叉注意力机制。我们的方法在10个基准测试中的11个上使用仅有的真实数据达到了最先进的性能，并且显著减少了参数数量和计算成本。
URL
https://arxiv.org/abs/2503.16184
