Abstract
Arabic handwritten text recognition (HTR) is challenging, especially for historical texts, due to diverse writing styles and the intrinsic features of Arabic script. Additionally, Arabic handwriting datasets are smaller compared to English ones, making it difficult to train generalizable Arabic HTR models. To address these challenges, we propose HATFormer, a transformer-based encoder-decoder architecture that builds on a state-of-the-art English HTR model. By leveraging the transformer's attention mechanism, HATFormer captures spatial contextual information to address the intrinsic challenges of Arabic script through differentiating cursive characters, decomposing visual representations, and identifying diacritics. Our customization to historical handwritten Arabic includes an image processor for effective ViT information preprocessing, a text tokenizer for compact Arabic text representation, and a training pipeline that accounts for a limited amount of historic Arabic handwriting data. HATFormer achieves a character error rate (CER) of 8.6% on the largest public historical handwritten Arabic dataset, with a 51% improvement over the best baseline in the literature. HATFormer also attains a comparable CER of 4.2% on the largest private non-historical dataset. Our work demonstrates the feasibility of adapting an English HTR method to a low-resource language with complex, language-specific challenges, contributing to advancements in document digitization, information retrieval, and cultural preservation.
Abstract (translated)
阿拉伯手写文本识别（HTR）具有挑战性，尤其是在历史文本中，因为阿拉伯文本具有多样性的书写风格和阿拉伯文字本特征。此外，阿拉伯手写数据集比英语数据集要小，这使得训练具有泛化能力的阿拉伯HTR模型变得困难。为了应对这些挑战，我们提出了HATFormer，一种基于最先进的英语HTR模型的Transformer编码器-解码器架构。通过利用Transformer的注意力机制，HATFormer捕捉到阿拉伯文本的空间上下文信息，通过区分手写字符、分解视觉表示和识别变体来解决阿拉伯文字本固有的挑战。我们对历史手写阿拉伯的定制包括一个有效的ViT信息预处理图像处理器、一个紧凑的阿拉伯文本词条izer和一个考虑有限历史阿拉伯手写数据训练工作流的训练管道。HATFormer在最大的公共历史手写阿拉伯数据集上的字符错误率（CER）为8.6%，在文献中的最佳基线上的性能提高了51%。HATFormer还在最大的私人非历史数据集上获得了与文献中类似且可比的CER，为将英语HTR方法应用于具有复杂、语言特定挑战的低资源语言奠定了基础，有助于促进文档数字化、信息检索和文化遗产保护的发展。
URL
https://arxiv.org/abs/2410.02179
