Abstract
Natural language processing (NLP) is a key technology to extract important patient information from clinical narratives to support healthcare applications. The rapid development of large language models (LLMs) has revolutionized many NLP tasks in the clinical domain, yet their optimal use in patient information extraction tasks requires further exploration. This study examines LLMs' effectiveness in patient information extraction, focusing on LLM architectures, fine-tuning strategies, and multi-task instruction tuning techniques for developing robust and generalizable patient information extraction systems. This study aims to explore key concepts of using LLMs for clinical concept and relation extraction tasks, including: (1) encoder-only or decoder-only LLMs, (2) prompt-based parameter-efficient fine-tuning (PEFT) algorithms, and (3) multi-task instruction tuning on few-shot learning performance. We benchmarked a suite of LLMs, including encoder-based LLMs (BERT, GatorTron) and decoder-based LLMs (GatorTronGPT, Llama 3.1, GatorTronLlama), across five datasets. We compared traditional full-size fine-tuning and prompt-based PEFT. We explored a multi-task instruction tuning framework that combines both tasks across four datasets to evaluate the zero-shot and few-shot learning performance using the leave-one-dataset-out strategy.
Abstract (translated)
自然语言处理（NLP）是提取临床叙述中重要患者信息以支持医疗应用的关键技术。大型语言模型（LLMs）的快速发展已经革新了临床领域中的许多NLP任务，但它们在患者信息提取任务中最优使用方法仍需进一步探索。本研究旨在考察LLMs在患者信息提取方面的有效性，重点关注LLM架构、微调策略以及多任务指令调整技术，以开发稳健且通用的患者信息抽取系统。该研究探讨了使用LLMs进行临床概念和关系提取任务的关键概念，包括：(1)仅编码器或仅解码器LLMs；(2)基于提示的参数高效微调（PEFT）算法；以及(3)在少量样本学习性能上的多任务指令调整技术。我们对一系列LLM进行了基准测试，其中包括基于编码器的模型（BERT、GatorTron）和基于解码器的模型（GatorTronGPT、Llama 3.1、GatorTronLlama），这些模型均在五个数据集上进行评估。我们对比了传统的全尺寸微调与基于提示的PEFT方法，并探索了一种多任务指令调整框架，该框架结合四个数据集中跨任务的任务，使用leave-one-dataset-out策略来评估零样本和少量样本学习性能。
URL
https://arxiv.org/abs/2509.04753
