Abstract
Retrieval-Augmented Generation (RAG) systems for biomedical literature are typically evaluated using ranking metrics like Mean Reciprocal Rank (MRR), which measure how well the system identifies the single most relevant chunk. We argue that for full-text scientific documents, this paradigm is incomplete: it rewards retrieval precision while ignoring retrieval breadth -- the ability to surface evidence from across a document's structural sections. We propose GraLC-RAG, a framework that unifies late chunking with graph-aware structural intelligence, introducing structure-aware chunk boundary detection, UMLS knowledge graph infusion, and graph-guided hybrid retrieval. We evaluate six strategies on 2,359 IMRaD-filtered PubMed Central articles using 2,033 cross-section questions and two metric families: standard ranking metrics (MRR, Recall@k) and structural coverage metrics (SecCov@k, CS Recall). Our results expose a sharp divergence: content-similarity methods achieve the highest MRR (0.517) but always retrieve from a single section, while structure-aware methods retrieve from up to 15.6x more sections. Generation experiments show that KG-infused retrieval narrows the answer-quality gap to delta-F1 = 0.009 while maintaining 4.6x section diversity. These findings demonstrate that standard metrics systematically undervalue structural retrieval and that closing the multi-section synthesis gap is a key open problem for biomedical RAG.
Abstract (translated)
用于生物医学文献的检索增强生成（RAG）系统通常采用平均倒数排名（MRR）等排序指标进行评估，这类指标衡量系统识别单个最相关文本块的能力。我们认为，对于全文科学文献，这一范式存在不足：它奖励检索精度，却忽视了检索广度——即从文档各结构章节中提取证据的能力。我们提出了GraLC-RAG框架，该框架将延迟分块与图感知结构智能相结合，引入了结构感知分块边界检测、UMLS知识图谱注入以及图引导混合检索。我们在2,359篇经IMRaD筛选的PubMed Central文章上评估了六种策略，使用2,033个跨章节问题以及两类指标：标准排序指标（MRR、Recall@k）和结构覆盖指标（SecCov@k、CS Recall）。我们的结果揭示了显著分化：内容相似性方法达到最高MRR（0.517），但始终仅从单一章节检索；而结构感知方法检索的章节数量高达前者的15.6倍。生成实验表明，知识图谱注入的检索将答案质量差距缩小至delta-F1 = 0.009，同时保持4.6倍的章节多样性。这些发现证明，标准指标系统性地低估了结构检索的价值，而弥合多章节综合的差距是生物医学RAG领域一个关键未决问题。
URL
https://arxiv.org/abs/2603.22633
