Abstract
Historical handwritten text recognition (HTR) is essential for unlocking the cultural and scholarly value of archival documents, yet digitization is often hindered by scarce transcriptions, linguistic variation, and highly diverse handwriting styles. In this study, we apply TrOCR, a state-of-the-art transformer-based HTR model, to 16th-century Latin manuscripts authored by Rudolf Gwalther. We investigate targeted image preprocessing and a broad suite of data augmentation techniques, introducing four novel augmentation methods designed specifically for historical handwriting characteristics. We also evaluate ensemble learning approaches to leverage the complementary strengths of augmentation-trained models. On the Gwalther dataset, our best single-model augmentation (Elastic) achieves a Character Error Rate (CER) of 1.86, while a top-5 voting ensemble achieves a CER of 1.60 - representing a 50% relative improvement over the best reported TrOCR_BASE result and a 42% improvement over the previous state of the art. These results highlight the impact of domain-specific augmentations and ensemble strategies in advancing HTR performance for historical manuscripts.
Abstract (translated)
历史手写文本识别（HTR）对于释放档案文献的文化和学术价值至关重要，然而由于缺乏转录、语言差异以及书写风格的高度多样性，数字化工作常常受到阻碍。在这项研究中，我们应用了TrOCR这一基于Transformer的最先进的HTR模型，对16世纪鲁道夫·格瓦尔特（Rudolf Gwalther）撰写的拉丁文手稿进行了处理。我们在图像预处理和一系列数据增强技术方面进行了探索，并引入了四种针对历史书写特征设计的新颖的数据增强方法。此外，我们还评估了集成学习策略，以利用经过数据增强训练的模型之间的互补优势。在Gwalther数据集上，我们的最佳单一模型增强（弹性变形）实现了1.86的字符错误率（CER），而前五投票集成则达到了1.60的CER——这一结果相较于此前报告的最佳TrOCR_BASE结果和最先进的方法分别提高了50%和42%。这些结果显示了领域特定的数据增强技术和集成策略在提升历史手稿HTR性能方面的影响。
URL
https://arxiv.org/abs/2508.11499
