Abstract
The primary challenge for handwriting recognition systems lies in managing long-range contextual dependencies, an issue that traditional models often struggle with. To mitigate it, attention mechanisms have recently been employed to enhance context-aware labelling, thereby achieving state-of-the-art performance. In the field of pattern recognition and image analysis, however, the use of contextual information in labelling problems has a long history and goes back at least to the early 1970's. Among the various approaches developed in those years, Relaxation Labelling (RL) processes have played a prominent role and have been the method of choice in the field for more than a decade. Contrary to recent transformer-based architectures, RL processes offer a principled approach to the use of contextual constraints, having a solid theoretic foundation grounded on variational inequality and game theory, as well as effective algorithms with convergence guarantees. In this paper, we propose a novel approach to handwriting recognition that integrates the strengths of two distinct methodologies. In particular, we propose integrating (trainable) RL processes with various well-established neural architectures and we introduce a sparsification technique that accelerates the convergence of the algorithm and enhances the overall system's performance. Experiments over several benchmark datasets show that RL processes can improve the generalisation ability, even surpassing in some cases transformer-based architectures.
Abstract (translated)
手写字符识别系统的主要挑战在于处理长距离上下文依赖关系，这是传统模型通常难以应对的问题。为解决这个问题，近年来采用了一种注意力机制来增强上下文敏感的标注，从而实现了最先进的性能。然而，在模式识别和图像分析领域，使用上下文信息进行标注问题已经有着悠久的历史，并且至少可以追溯到20世纪70年代初。在那几年里开发的各种方法中， relaxation标记（RL）过程发挥了重要作用，并且已经成为该领域的首选方法超过十年。与最近基于Transformer的架构不同，RL过程具有基于变分不等式和博弈论的严谨的理论基础，以及具有收敛保证的有效算法。在本文中，我们提出了一种将两个不同的方法论相结合的新型手写字符识别方法。特别地，我们提出将可训练的RL过程与各种已经确立的神经网络架构集成，并引入了一种稀疏化技术，加速算法的收敛，提高整体系统的性能。在多个基准数据集上的实验结果表明，RL过程可以提高泛化能力，甚至在某些情况下超过了基于Transformer的架构。
URL
https://arxiv.org/abs/2409.05699
