Abstract
Extracting medication names from handwritten doctor prescriptions is challenging due to the wide variability in handwriting styles and prescription formats. This paper presents a robust method for extracting medicine names using a combination of Mask R-CNN and Transformer-based Optical Character Recognition (TrOCR) with Multi-Head Attention and Positional Embeddings. A novel dataset, featuring diverse handwritten prescriptions from various regions of Pakistan, was utilized to fine-tune the model on different handwriting styles. The Mask R-CNN model segments the prescription images to focus on the medicinal sections, while the TrOCR model, enhanced by Multi-Head Attention and Positional Embeddings, transcribes the isolated text. The transcribed text is then matched against a pre-existing database for accurate identification. The proposed approach achieved a character error rate (CER) of 1.4% on standard benchmarks, highlighting its potential as a reliable and efficient tool for automating medicine name extraction.
Abstract (translated)
从手写医生处方中提取药品名称具有挑战性，因为书写风格和处方格式的多样性。本文介绍了一种结合使用Mask R-CNN和基于Transformer的光学字符识别（TrOCR）的方法来提取药品名称，该方法采用了多头注意力机制和位置嵌入技术。研究中利用了一个新数据集，其中包含了来自巴基斯坦不同地区的多样化手写处方，用以调整模型以适应不同的书写风格。Mask R-CNN模型用于分割处方图像，专注于药物部分，而由多头注意力机制和位置嵌入增强的TrOCR模型则负责转录孤立文本。随后将转录后的文本与现有的数据库进行匹配，以实现准确识别。所提出的方法在标准基准测试中达到了1.4%的字符错误率（CER），显示出其作为自动提取药品名称可靠而高效的工具的巨大潜力。
URL
https://arxiv.org/abs/2412.18199
