Abstract
Recent advancements in handwritten text recognition (HTR) have enabled the effective conversion of handwritten text to digital formats. However, achieving robust recognition across diverse writing styles remains challenging. Traditional HTR methods lack writer-specific personalization at test time due to limitations in model architecture and training strategies. Existing attempts to bridge this gap, through gradient-based meta-learning, still require labeled examples and suffer from parameter-inefficient fine-tuning, leading to substantial computational and memory overhead. To overcome these challenges, we propose an efficient framework that formulates personalization as prompt tuning, incorporating an auxiliary image reconstruction task with a self-supervised loss to guide prompt adaptation with unlabeled test-time examples. To ensure self-supervised loss effectively minimizes text recognition error, we leverage meta-learning to learn the optimal initialization of the prompts. As a result, our method allows the model to efficiently capture unique writing styles by updating less than 1% of its parameters and eliminating the need for time-intensive annotation processes. We validate our approach on the RIMES and IAM Handwriting Database benchmarks, where it consistently outperforms previous state-of-the-art methods while using 20x fewer parameters. We believe this represents a significant advancement in personalized handwritten text recognition, paving the way for more reliable and practical deployment in resource-constrained scenarios.
Abstract (translated)
最近在手写文本识别（HTR）领域的进展使得将手写文本转换为数字格式成为可能。然而，实现不同书写风格下的稳健识别仍然具有挑战性。传统的方法由于模型架构和训练策略的限制，在测试时缺乏特定于书写的个性化调整。尽管有一些尝试通过基于梯度的元学习来弥合这一差距，但这些方法仍需要标注示例，并且因参数效率低下的微调而遭受计算量大和内存开销大的问题。 为了解决这些问题，我们提出了一种高效的框架，该框架将个性化视为提示调整，并结合了辅助图像重建任务以及自监督损失来引导未标记的测试时间实例中的提示适应。为了确保自监督损失能有效减少文本识别误差，我们利用元学习来学习提示的最佳初始设置。因此，我们的方法使模型能够通过更新不到1%的参数高效地捕捉独特的书写风格，并消除了耗时的标注过程需求。 我们在RIMES和IAM手写数据库基准测试上验证了这一方法的有效性，在这些测试中，它持续优于先前最先进的方法，同时使用的参数减少了20倍。我们认为这代表了个性化手写文本识别的重要进展，为资源受限场景中的更可靠和实际部署铺平了道路。
URL
https://arxiv.org/abs/2505.20513
