Abstract
Mask Diffusion Models (MDMs) have recently emerged as a promising alternative to auto-regressive models (ARMs) for vision-language tasks, owing to their flexible balance of efficiency and accuracy. In this paper, for the first time, we introduce MDMs into the Scene Text Recognition (STR) task. We show that vanilla MDM lags behind ARMs in terms of accuracy, although it improves recognition efficiency. To bridge this gap, we propose MDiff4STR, a Mask Diffusion model enhanced with two key improvement strategies tailored for STR. Specifically, we identify two key challenges in applying MDMs to STR: noising gap between training and inference, and overconfident predictions during inference. Both significantly hinder the performance of MDMs. To mitigate the first issue, we develop six noising strategies that better align training with inference behavior. For the second, we propose a token-replacement noise mechanism that provides a non-mask noise type, encouraging the model to reconsider and revise overly confident but incorrect predictions. We conduct extensive evaluations of MDiff4STR on both standard and challenging STR benchmarks, covering diverse scenarios including irregular, artistic, occluded, and Chinese text, as well as whether the use of pretraining. Across these settings, MDiff4STR consistently outperforms popular STR models, surpassing state-of-the-art ARMs in accuracy, while maintaining fast inference with only three denoising steps. Code: this https URL.
Abstract (translated)
最近，Mask Diffusion Models (MDM) 作为一种有前景的替代方案出现了，它可以为视觉-语言任务提供比自回归模型（ARM）更加灵活的效率和准确性平衡。在本文中，我们首次将MDM引入到Scene Text Recognition (STR) 任务中。研究表明，在提高识别效率的同时，标准的MDM在准确率上仍然落后于ARM。为了弥补这一差距，我们提出了MDiff4STR，这是一种增强型Mask Diffusion模型，它采用了两种针对STR优化的关键改进策略。 具体来说，我们在将MDMs应用于STR时确定了两个关键挑战：训练和推理之间的噪音差异，以及推理过程中过度自信的预测问题。这两个因素都显著阻碍了MDM的表现。为了解决第一个问题，我们开发出了六种噪音策略，以更好地协调训练与推理行为的一致性。对于第二个问题，则提出了一种基于标记替换的噪声机制来提供一种非屏蔽类型的噪声，促使模型重新考虑并修正那些过度自信但实际上错误的预测。 我们在标准和具有挑战性的STR基准测试上对MDiff4STR进行了广泛的评估，涵盖了包括不规则、艺术化、被遮挡的文字以及中文文本在内的多种场景，并探讨了预训练的影响。在这些设置中，MDiff4STR始终超越流行的STR模型，在准确性方面超过了最先进的ARMs，同时通过仅使用三个去噪步骤保持了快速的推理速度。 该研究的代码可在此链接获取：[请插入实际链接]
URL
https://arxiv.org/abs/2512.01422
