Abstract
While state-of-the-art Handwritten Text Recognition (HTR) models perform well on standard benchmarks, they frequently struggle with writers exhibiting highly specific styles that are underrepresented in the training data. To handle unseen and atypical writers, writer adaptation techniques personalize HTR models to individual handwriting styles. Leading writer adaptation methods require either offline fine-tuning or parameter updates at inference time, both involving gradient computation and backpropagation, which increase computational costs and demand careful hyperparameter tuning. In this work, we propose a novel context-driven HTR framework3 inspired by multimodal in-context learning, enabling inference-time writer adaptation using only a few examples from the target writer without any parameter updates. We further demonstrate the impact of context length, design a compact 8M-parameter CNN-Transformer that enables few-shot in-context adaptation, and show that combining context-driven and standard OCR training strategies leads to complementary improvements. Experiments on IAM and RIMES validate our approach with Character Error Rates of 3.92% and 2.34%, respectively, surpassing all writer-independent HTR models without requiring any parameter updates at inference time.
Abstract (translated)
尽管最先进的手写文本识别（HTR）模型在标准基准测试中表现良好，但它们经常难以处理在训练数据中代表性不足的高度特定书写风格。为应对未见且非典型的书写者，书写者自适应技术能够将HTR模型个性化以适应个体书写风格。领先的书写者自适应方法需要离线微调或在推理时进行参数更新，两者都涉及梯度计算和反向传播，这会增加计算成本并需要仔细的超参数调优。在本研究中，我们提出了一种新颖的、受多模态上下文学习启发的上下文驱动HTR框架，能够在推理时仅使用目标书写者的少量示例实现书写者自适应，且无需任何参数更新。我们进一步展示了上下文长度的影响，设计了一个紧凑的8M参数CNN-Transformer模型以实现少样本上下文自适应，并证明结合上下文驱动与标准OCR训练策略能带来互补性改进。在IAM和RIMES数据集上的实验验证了我们的方法，分别达到3.92%和2.34%的字符错误率，超越了所有无需在推理时进行参数更新的书写者无关HTR模型。
URL
https://arxiv.org/abs/2603.29450
