Abstract
Large language models have demonstrated remarkable capabilities in biomedical natural language processing, yet their rapid growth in size and computational requirements present a major barrier to adoption in healthcare settings where data privacy precludes cloud deployment and resources are limited. In this study, we systematically evaluated the impact of quantization on 12 state-of-the-art large language models, including both general-purpose and biomedical-specific models, across eight benchmark datasets covering four key tasks: named entity recognition, relation extraction, multi-label classification, and question answering. We show that quantization substantially reduces GPU memory requirements-by up to 75%-while preserving model performance across diverse tasks, enabling the deployment of 70B-parameter models on 40GB consumer-grade GPUs. In addition, domain-specific knowledge and responsiveness to advanced prompting methods are largely maintained. These findings provide significant practical and guiding value, highlighting quantization as a practical and effective strategy for enabling the secure, local deployment of large yet high-capacity language models in biomedical contexts, bridging the gap between technical advances in AI and real-world clinical translation.
Abstract (translated)
大型语言模型在生物医学自然语言处理方面展示了非凡的能力，但其规模迅速扩大和计算需求增加，在数据隐私限制云部署且资源有限的医疗环境中构成了主要障碍。在这项研究中，我们系统地评估了量化对12种最先进的大规模语言模型（包括通用型和生物医学专用型）的影响，这些模型在涵盖四个关键任务的八个基准数据集上进行了测试：命名实体识别、关系抽取、多标签分类和问答。研究表明，量化技术能够显著减少GPU内存需求（最高可达75%），同时保持模型在多种任务上的性能，使得拥有700亿参数的大规模模型能够在40GB消费级GPU上运行。此外，领域特定知识以及对先进提示方法的响应性也得到了很大程度的保留。 这些发现提供了重要的实用和指导价值，突显了量化作为使大型且高性能语言模型在生物医学环境中实现安全本地部署的一种实际有效策略的重要性，从而弥合了人工智能技术进步与现实世界临床应用之间的差距。
URL
https://arxiv.org/abs/2509.04534
