Abstract
Handwritten Text Recognition (HTR) is essential for document analysis and digitization. However, handwritten data often contains user-identifiable information, such as unique handwriting styles and personal lexicon choices, which can compromise privacy and erode trust in AI services. Legislation like the ``right to be forgotten'' underscores the necessity for methods that can expunge sensitive information from trained models. Machine unlearning addresses this by selectively removing specific data from models without necessitating complete retraining. Yet, it frequently encounters a privacy-accuracy tradeoff, where safeguarding privacy leads to diminished model performance. In this paper, we introduce a novel two-stage unlearning strategy for a multi-head transformer-based HTR model, integrating pruning and random labeling. Our proposed method utilizes a writer classification head both as an indicator and a trigger for unlearning, while maintaining the efficacy of the recognition head. To our knowledge, this represents the first comprehensive exploration of machine unlearning within HTR tasks. We further employ Membership Inference Attacks (MIA) to evaluate the effectiveness of unlearning user-identifiable information. Extensive experiments demonstrate that our approach effectively preserves privacy while maintaining model accuracy, paving the way for new research directions in the document analysis community. Our code will be publicly available upon acceptance.
Abstract (translated)
手写文本识别（HTR）在文档分析和数字化中至关重要。然而，手写数据常常包含可辨识个人的信息，例如独特的书写风格和个人词汇选择，这可能危及隐私并削弱用户对AI服务的信任。类似于“被遗忘权”这样的立法强调了从训练模型中清除敏感信息的方法的必要性。机器卸载通过有选择地从模型中移除特定数据而不必重新进行全面训练来解决这个问题。然而，在保护隐私和保证准确性之间，经常存在一种折衷，即保护隐私会导致模型性能下降。 本文介绍了针对基于多头变换器的手写文本识别（HTR）模型的一种新的两阶段卸载策略，该策略结合了修剪和随机标签处理。我们提出的方法利用书写者分类头作为卸载的指标和触发器，同时保持识别头的有效性。据我们所知，这是首次全面探讨在HTR任务中机器卸载的研究。 此外，我们使用成员资格推理攻击（MIA）来评估删除可辨识用户信息的效果。广泛的实验表明，我们的方法能够有效地保护隐私的同时保持模型的准确性，为文档分析社区的新研究方向铺平了道路。我们的代码将在接受后公开提供。
URL
https://arxiv.org/abs/2504.08616
