Relation_Extraction
-
Integrating Causal Reasoning into Automated Fact-Checking
2025-12-15 12:56:00Youssra Rebboud, Pasquale Lisena, Raphael TroncyAbstract
In fact-checking applications, a common reason to reject a claim is to detect the presence of erroneous cause-effect relationships between the events at play. However, current automated fact-checking methods lack dedicated causal-based reasoning, potentially missing a valuable opportunity for semantically rich explainability. To address this gap, we propose a methodology that combines event relation extraction, semantic similarity computation, and rule-based reasoning to detect logical inconsistencies between chains of events mentioned in a claim and in an evidence. Evaluated on two fact-checking datasets, this method establishes the first baseline for integrating fine-grained causal event relationships into fact-checking and enhance explainability of verdict prediction.
Abstract (translated)
在事实核查应用中，拒绝一个声明的常见原因之一是检测事件之间是否存在错误的因果关系。然而，当前的自动化事实核查方法缺乏专门基于因果关系的推理能力，这可能错失了提高语义丰富解释性的宝贵机会。为了填补这一空白，我们提出了一种结合事件关系提取、语义相似性计算和规则基础推理的方法，以检测声明中提及的一系列事件与证据中提到的事件之间的逻辑不一致。在两个事实核查数据集上进行评估后，这种方法建立了将细粒度因果事件关系集成到事实核查中的第一个基准，并提高了判决预测的解释性。
URL
https://arxiv.org/abs/2512.13286
PDF
-
LexRel: Benchmarking Legal Relation Extraction for Chinese Civil Cases
2025-12-14 11:16:39Yida Cai, Ranjuexiao Hu, Huiyuan Xie, Chenyang Li, Yun Liu, Yuxiao Ye, Zhenghao Liu, Weixing Shen, Zhiyuan LiuAbstract
Legal relations form a highly consequential analytical framework of civil law system, serving as a crucial foundation for resolving disputes and realizing values of the rule of law in judicial practice. However, legal relations in Chinese civil cases remain underexplored in the field of legal artificial intelligence (legal AI), largely due to the absence of comprehensive schemas. In this work, we firstly introduce a comprehensive schema, which contains a hierarchical taxonomy and definitions of arguments, for AI systems to capture legal relations in Chinese civil cases. Based on this schema, we then formulate legal relation extraction task and present LexRel, an expert-annotated benchmark for legal relation extraction in Chinese civil law. We use LexRel to evaluate state-of-the-art large language models (LLMs) on legal relation extractions, showing that current LLMs exhibit significant limitations in accurately identifying civil legal relations. Furthermore, we demonstrate that incorporating legal relations information leads to consistent performance gains on other downstream legal AI tasks.
Abstract (translated)
法律关系构成了民法体系中一个高度重要的分析框架，是解决纠纷和实现法治价值的重要基础。然而，在中国民事案件中，由于缺乏全面的架构，法律关系在法律人工智能（legal AI）领域仍被较少研究。在这项工作中，我们首先介绍了一个全面的架构，包括法律关系的分层分类法及其定义，使AI系统能够捕捉到中国民事案件中的法律关系。基于这一架构，我们将法律关系抽取任务具体化，并提出了LexRel，一个专家标注的数据集，作为中文民法领域中法律关系提取的基准。我们使用LexRel来评估当前最先进的大型语言模型（LLMs）在法律关系抽取上的表现，显示出现有LLMs在准确识别民事法律关系方面存在显著局限性。此外，我们还证明了将法律关系信息融入其中可以持续提升其他下游法律人工智能任务的表现。
URL
https://arxiv.org/abs/2512.12643
PDF
-
SCIR: A Self-Correcting Iterative Refinement Framework for Enhanced Information Extraction Based on Schema
2025-12-13 14:07:25Yushen Fang, Jianjun Li, Mingqian Ding, Chang Liu, Xinchi Zou, Wenqi YangAbstract
Although Large language Model (LLM)-powered information extraction (IE) systems have shown impressive capabilities, current fine-tuning paradigms face two major limitations: high training costs and difficulties in aligning with LLM preferences. To address these issues, we propose a novel universal IE paradigm, the Self-Correcting Iterative Refinement (SCIR) framework, along with a Multi-task Bilingual (Chinese-English) Self-Correcting (MBSC) dataset containing over 100,000 entries. The SCIR framework achieves plug-and-play compatibility with existing LLMs and IE systems through its Dual-Path Self-Correcting module and feedback-driven optimization, thereby significantly reducing training costs. Concurrently, the MBSC dataset tackles the challenge of preference alignment by indirectly distilling GPT-4's capabilities into IE result detection models. Experimental results demonstrate that SCIR outperforms state-of-the-art IE methods across three key tasks: named entity recognition, relation extraction, and event extraction, achieving a 5.27 percent average improvement in span-based Micro-F1 while reducing training costs by 87 percent compared to baseline approaches. These advancements not only enhance the flexibility and accuracy of IE systems but also pave the way for lightweight and efficient IE paradigms.
Abstract (translated)
尽管大型语言模型（LLM）驱动的信息抽取（IE）系统展现了令人印象深刻的能力，但当前的微调范式面临着两个主要限制：高昂的训练成本和与LLM偏好对齐的困难。为了解决这些问题，我们提出了一种新型的通用信息抽取框架——自我校正迭代精炼（Self-Correcting Iterative Refinement, SCIR）以及一个多任务双语（中英文）自我校正数据集MBSC，包含超过10万条记录。通过其双路径自我校正模块和反馈驱动优化，SCIR框架实现了与现有LLM和IE系统的即插即用兼容性，从而大幅降低了训练成本。同时，MBSC数据集通过间接提炼GPT-4的能力到信息抽取结果检测模型中，解决了偏好对齐的挑战。 实验结果显示，在命名实体识别、关系提取和事件抽取三项关键任务上，SCIR方法超越了现有的最先进的IE方法，平均提高了基于跨度的Micro-F1分数5.27%，并且相比基准方法将训练成本降低了87%。这些进展不仅提升了信息抽取系统的灵活性和准确性，还为轻量级且高效的IE范式铺平了道路。
URL
https://arxiv.org/abs/2512.12337
PDF
-
PharmaShip: An Entity-Centric, Reading-Order-Supervised Benchmark for Chinese Pharmaceutical Shipping Documents
2025-11-29 06:55:45Tingwei Xie, Tianyi Zhou, Yonghong SongAbstract
We present PharmaShip, a real-world Chinese dataset of scanned pharmaceutical shipping documents designed to stress-test pre-trained text-layout models under noisy OCR and heterogeneous templates. PharmaShip covers three complementary tasks-sequence entity recognition (SER), relation extraction (RE), and reading order prediction (ROP)-and adopts an entity-centric evaluation protocol to minimize confounds across architectures. We benchmark five representative baselines spanning pixel-aware and geometry-aware families (LiLT, LayoutLMv3-base, GeoLayoutLM and their available RORE-enhanced variants), and standardize preprocessing, splits, and optimization. Experiments show that pixels and explicit geometry provide complementary inductive biases, yet neither alone is sufficient: injecting reading-order-oriented regularization consistently improves SER and EL and yields the most robust configuration, while longer positional coverage stabilizes late-page predictions and reduces truncation artifacts. ROP is accurate at the word level but challenging at the segment level, reflecting boundary ambiguity and long-range crossings. PharmaShip thus establishes a controlled, reproducible benchmark for safety-critical document understanding in the pharmaceutical domain and highlights sequence-aware constraints as a transferable bias for structure modeling. We release the dataset at this https URL.
Abstract (translated)
我们介绍了PharmaShip，这是一个包含扫描的中国药品运输文件的真实世界数据集，旨在测试预训练文本布局模型在噪声OCR和异构模板下的性能。PharmaShip涵盖了三个互补的任务：序列实体识别（SER）、关系抽取（RE）以及阅读顺序预测（ROP），并采用以实体为中心的评估协议来最小化不同架构之间的混淆因素。我们对五种具有代表性的基线方法进行了基准测试，这些方法跨越像素感知和几何感知两大类（包括LiLT、LayoutLMv3-base、GeoLayoutLM及其可用的RORE增强版本），并且统一了预处理、数据拆分以及优化标准。 实验结果表明，像素信息与显式几何结构提供了互补性的归纳偏置，但两者单独使用都不足以达到最佳效果：注入以阅读顺序为导向的正则化能够持续改进SER和EL性能，并提供最稳健的配置；而增加位置覆盖范围可以稳定页面后段的预测并减少截断产生的伪影。尽管ROP在单词级别上的准确性较高，但在分段级别上却具有挑战性，这反映了边界模糊以及长距离跨越的问题。 因此，PharmaShip为药品领域中关键安全性的文件理解建立了可控制且可重复的基准测试，并强调了序列感知约束作为结构建模中的可转移偏置。数据集可在以下链接获取：[此链接](this https URL)。
URL
https://arxiv.org/abs/2512.23714
PDF
-
Applying Relation Extraction and Graph Matching to Answering Multiple Choice Questions
2025-11-18 05:03:27Naoki Shimoda, Akihiro YamamotoAbstract
In this research, we combine Transformer-based relation extraction with matching of knowledge graphs (KGs) and apply them to answering multiple-choice questions (MCQs) while maintaining the traceability of the output process. KGs are structured representations of factual knowledge consisting of entities and relations. Due to the high construction cost, they had been regarded as static databases with validated links. However, the recent development of Transformer-based relation extraction (RE) methods has enabled us to generate KGs dynamically by giving them natural language texts, and thereby opened the possibility for representing the meaning of the input sentences with the created KGs. Using this effect, we propose a method that answers MCQs in the "fill-in-the-blank" format, taking care of the point that RE methods generate KGs that represent false information if provided with factually incorrect texts. We measure the truthfulness of each question sentence by (i) converting the sentence into a relational graph using an RE method and (ii) verifying it against factually correct KGs under the closed-world assumption. The experimental results demonstrate that our method correctly answers up to around 70% of the questions, while providing traceability of the procedure. We also highlight that the question category has a vast influence on the accuracy.
Abstract (translated)
URL
https://arxiv.org/abs/2511.14144
PDF
-
BeDiscovER: The Benchmark of Discourse Understanding in the Era of Reasoning Language Models
2025-11-17 07:50:12Chuyuan Li, Giuseppe CareniniAbstract
We introduce BeDiscovER (Benchmark of Discourse Understanding in the Era of Reasoning Language Models), an up-to-date, comprehensive suite for evaluating the discourse-level knowledge of modern LLMs. BeDiscovER compiles 5 publicly available discourse tasks across discourse lexicon, (multi-)sentential, and documental levels, with in total 52 individual datasets. It covers both extensively studied tasks such as discourse parsing and temporal relation extraction, as well as some novel challenges such as discourse particle disambiguation (e.g., ``just''), and also aggregates a shared task on Discourse Relation Parsing and Treebanking for multilingual and multi-framework discourse relation classification. We evaluate open-source LLMs: Qwen3 series, DeepSeek-R1, and frontier model such as GPT-5-mini on BeDiscovER, and find that state-of-the-art models exhibit strong performance in arithmetic aspect of temporal reasoning, but they struggle with full document reasoning and some subtle semantic and discourse phenomena, such as rhetorical relation recognition.
Abstract (translated)
URL
https://arxiv.org/abs/2511.13095
PDF
-
GraphIF: Enhancing Multi-Turn Instruction Following for Large Language Models with Relation Graph Prompt
2025-11-13 07:49:38Zhenhe Li, Can Lin, Ling Zheng, Wen-Da Wei, Junli Liang, Qi SongAbstract
Multi-turn instruction following is essential for building intelligent conversational systems that can consistently adhere to instructions across dialogue turns. However, existing approaches to enhancing multi-turn instruction following primarily rely on collecting or generating large-scale multi-turn dialogue datasets to fine-tune large language models (LLMs), which treat each response generation as an isolated task and fail to explicitly incorporate multi-turn instruction following into the optimization objectives. As a result, instruction-tuned LLMs often struggle with complex long-distance constraints. In multi-turn dialogues, relational constraints across turns can be naturally modeled as labeled directed edges, making graph structures particularly suitable for modeling multi-turn instruction following. Despite this potential, leveraging graph structures to enhance the multi-turn instruction following capabilities of LLMs remains unexplored. To bridge this gap, we propose GraphIF, a plug-and-play framework that models multi-turn dialogues as directed relation graphs and leverages graph prompts to enhance the instruction following capabilities of LLMs. GraphIF comprises three key components: (1) an agent-based relation extraction module that captures inter-turn semantic relations via action-triggered mechanisms to construct structured graphs; (2) a relation graph prompt generation module that converts structured graph information into natural language prompts; and (3) a response rewriting module that refines initial LLM outputs using the generated graph prompts. Extensive experiments on two long multi-turn dialogue datasets demonstrate that GraphIF can be seamlessly integrated into instruction-tuned LLMs and leads to significant improvements across all four multi-turn instruction-following evaluation metrics.
Abstract (translated)
URL
https://arxiv.org/abs/2511.10051
PDF
-
GSAP-ERE: Fine-Grained Scholarly Entity and Relation Extraction Focused on Machine Learning
2025-11-12 15:26:07Wolfgang Otto, Lu Gan, Sharmila Upadhyaya, Saurav Karmakar, Stefan DietzeAbstract
Research in Machine Learning (ML) and AI evolves rapidly. Information Extraction (IE) from scientific publications enables to identify information about research concepts and resources on a large scale and therefore is a pathway to improve understanding and reproducibility of ML-related research. To extract and connect fine-grained information in ML-related research, e.g. method training and data usage, we introduce GSAP-ERE. It is a manually curated fine-grained dataset with 10 entity types and 18 semantically categorized relation types, containing mentions of 63K entities and 35K relations from the full text of 100 ML publications. We show that our dataset enables fine-tuned models to automatically extract information relevant for downstream tasks ranging from knowledge graph (KG) construction, to monitoring the computational reproducibility of AI research at scale. Additionally, we use our dataset as a test suite to explore prompting strategies for IE using Large Language Models (LLM). We observe that the performance of state-of-the-art LLM prompting methods is largely outperformed by our best fine-tuned baseline model (NER: 80.6%, RE: 54.0% for the fine-tuned model vs. NER: 44.4%, RE: 10.1% for the LLM). This disparity of performance between supervised models and unsupervised usage of LLMs suggests datasets like GSAP-ERE are needed to advance research in the domain of scholarly information extraction.
Abstract (translated)
URL
https://arxiv.org/abs/2511.09411
PDF
-
Relation as a Prior: A Novel Paradigm for LLM-based Document-level Relation Extraction
2025-11-11 11:55:27Qiankun Pi, Yepeng Sun, Jicang Lu, Qinlong Fan, Ningbo Huang, Shiyu WangAbstract
Large Language Models (LLMs) have demonstrated their remarkable capabilities in document understanding. However, recent research reveals that LLMs still exhibit performance gaps in Document-level Relation Extraction (DocRE) as requiring fine-grained comprehension. The commonly adopted "extract entities then predict relations" paradigm in LLM-based methods leads to these gaps due to two main reasons: (1) Numerous unrelated entity pairs introduce noise and interfere with the relation prediction for truly related entity pairs. (2) Although LLMs have identified semantic associations between entities, relation labels beyond the predefined set are still treated as prediction errors. To address these challenges, we propose a novel Relation as a Prior (RelPrior) paradigm for LLM-based DocRE. For challenge (1), RelPrior utilizes binary relation as a prior to extract and determine whether two entities are correlated, thereby filtering out irrelevant entity pairs and reducing prediction noise. For challenge (2), RelPrior utilizes predefined relation as a prior to match entities for triples extraction instead of directly predicting relation. Thus, it avoids misjudgment caused by strict predefined relation labeling. Extensive experiments on two benchmarks demonstrate that RelPrior achieves state-of-the-art performance, surpassing existing LLM-based methods.
Abstract (translated)
URL
https://arxiv.org/abs/2511.08143
PDF
-
A systematic review of relation extraction task since the emergence of Transformers
2025-11-05 16:28:48Ringwald Celian, Gandon, Fabien, Faron Catherine, Michel Franck, Abi Akl HannaAbstract
This article presents a systematic review of relation extraction (RE) research since the advent of Transformer-based models. Using an automated framework to collect and annotate publications, we analyze 34 surveys, 64 datasets, and 104 models published between 2019 and 2024. The review highlights methodological advances, benchmark resources, and the integration of semantic web technologies. By consolidating results across multiple dimensions, the study identifies current trends, limitations, and open challenges, offering researchers and practitioners a comprehensive reference for understanding the evolution and future directions of RE.
Abstract (translated)
URL
https://arxiv.org/abs/2511.03610
PDF
-
Kastor: Fine-tuned Small Language Models for Shape-based Active Relation Extraction
2025-11-05 13:43:47Ringwald Celian, Gandon Fabien, Faron Catherine, Michel Franck, Abi Akl HannaAbstract
RDF pattern-based extraction is a compelling approach for fine-tuning small language models (SLMs) by focusing a relation extraction task on a specified SHACL shape. This technique enables the development of efficient models trained on limited text and RDF data. In this article, we introduce Kastor, a framework that advances this approach to meet the demands for completing and refining knowledge bases in specialized domains. Kastor reformulates the traditional validation task, shifting from single SHACL shape validation to evaluating all possible combinations of properties derived from the shape. By selecting the optimal combination for each training example, the framework significantly enhances model generalization and performance. Additionally, Kastor employs an iterative learning process to refine noisy knowledge bases, enabling the creation of robust models capable of uncovering new, relevant facts
Abstract (translated)
URL
https://arxiv.org/abs/2511.03466
PDF
-
Overcoming the Generalization Limits of SLM Finetuning for Shape-Based Extraction of Datatype and Object Properties
2025-11-05 12:16:51C\'elian Ringwald, Fabien Gandon, Catherine Faron, Franck Michel, Hanna Abi AklAbstract
Small language models (SLMs) have shown promises for relation extraction (RE) when extracting RDF triples guided by SHACL shapes focused on common datatype properties. This paper investigates how SLMs handle both datatype and object properties for a complete RDF graph extraction. We show that the key bottleneck is related to long-tail distribution of rare properties. To solve this issue, we evaluate several strategies: stratified sampling, weighted loss, dataset scaling, and template-based synthetic data augmentation. We show that the best strategy to perform equally well over unbalanced target properties is to build a training set where the number of occurrences of each property exceeds a given threshold. To enable reproducibility, we publicly released our datasets, experimental results and code. Our findings offer practical guidance for training shape-aware SLMs and highlight promising directions for future work in semantic RE.
Abstract (translated)
URL
https://arxiv.org/abs/2511.03407
PDF
-
DREaM: Drug-Drug Relation Extraction via Transfer Learning Method
2025-10-27 10:27:00Ali Fata, Hossein Rahmani, Parinaz Soltanzadeh, Amirhossein Derakhshan, Behrouz Minaei BidgoliAbstract
Relation extraction between drugs plays a crucial role in identifying drug drug interactions and predicting side effects. The advancement of machine learning methods in relation extraction, along with the development of large medical text databases, has enabled the low cost extraction of such relations compared to other approaches that typically require expert knowledge. However, to the best of our knowledge, there are limited datasets specifically designed for drug drug relation extraction currently available. Therefore, employing transfer learning becomes necessary to apply machine learning methods in this domain. In this study, we propose DREAM, a method that first employs a trained relation extraction model to discover relations between entities and then applies this model to a corpus of medical texts to construct an ontology of drug relationships. The extracted relations are subsequently validated using a large language model. Quantitative results indicate that the LLM agreed with 71 of the relations extracted from a subset of PubMed abstracts. Furthermore, our qualitative analysis indicates that this approach can uncover ambiguities in the medical domain, highlighting the challenges inherent in relation extraction in this field.
Abstract (translated)
URL
https://arxiv.org/abs/2510.23189
PDF
-
Combining Distantly Supervised Models with In Context Learning for Monolingual and Cross-Lingual Relation Extraction
2025-10-21 06:55:19Vipul Rathore, Malik Hammad Faisal, Parag Singla, MausamAbstract
Distantly Supervised Relation Extraction (DSRE) remains a long-standing challenge in NLP, where models must learn from noisy bag-level annotations while making sentence-level predictions. While existing state-of-the-art (SoTA) DSRE models rely on task-specific training, their integration with in-context learning (ICL) using large language models (LLMs) remains underexplored. A key challenge is that the LLM may not learn relation semantics correctly, due to noisy annotation. In response, we propose HYDRE -- HYbrid Distantly Supervised Relation Extraction framework. It first uses a trained DSRE model to identify the top-k candidate relations for a given test sentence, then uses a novel dynamic exemplar retrieval strategy that extracts reliable, sentence-level exemplars from training data, which are then provided in LLM prompt for outputting the final relation(s). We further extend HYDRE to cross-lingual settings for RE in low-resource languages. Using available English DSRE training data, we evaluate all methods on English as well as a newly curated benchmark covering four diverse low-resource Indic languages -- Oriya, Santali, Manipuri, and Tulu. HYDRE achieves up to 20 F1 point gains in English and, on average, 17 F1 points on Indic languages over prior SoTA DSRE models. Detailed ablations exhibit HYDRE's efficacy compared to other prompting strategies.
Abstract (translated)
远监督关系抽取（DSRE）在自然语言处理领域一直是一个长期存在的挑战，模型需要从带有噪声的句子级别标注中学习并进行句级别的预测。尽管现有的最先进的（SoTA）DSRE模型依赖于特定任务的训练，但它们与大规模语言模型（LLMs）中的上下文学习（ICL）集成的研究却相对较少。一个关键的问题是由于标注数据的噪声性，LLM可能无法正确地学习关系语义。为了解决这个问题，我们提出了HYDRE——混合远监督关系抽取框架。该框架首先使用经过训练的DSRE模型来识别给定测试句子中的前k个候选关系，然后采用一种新颖的动态示例检索策略从训练数据中提取可靠、句级别的示例，并将这些示例如实提供在LLM提示中以输出最终的关系。我们进一步扩展了HYDRE框架的应用范围，使其适用于低资源语言的关系抽取跨语言场景。 使用现有的英文DSRE训练数据，我们在英语以及一个新的基准测试上评估了所有方法——该测试涵盖了四种多样的低资源印地语系语言：奥里亚语、桑塔利语、梅泰语和图鲁语。HYDRE在英语上的表现比之前的SoTA DSRE模型高出最多20个F1分数点，并且平均而言，在印地语系语言上也提高了17个F1分数点。详细的消融实验展示了HYDRE相较于其他提示策略的优越性。
URL
https://arxiv.org/abs/2510.18344
PDF
-
Are Large Language Models Effective Knowledge Graph Constructors?
2025-10-13 11:37:48Ruirui Chen, Weifeng Jiang, Chengwei Qin, Bo Xiong, Fiona Liausvia, Dongkyu Choi, Boon Kiat QuekAbstract
Knowledge graphs (KGs) are vital for knowledge-intensive tasks and have shown promise in reducing hallucinations in large language models (LLMs). However, constructing high-quality KGs remains difficult, requiring accurate information extraction and structured representations that support interpretability and downstream utility. Existing LLM-based approaches often focus narrowly on entity and relation extraction, limiting coverage to sentence-level contexts or relying on predefined schemas. We propose a hierarchical extraction framework that organizes information at multiple levels, enabling the creation of semantically rich and well-structured KGs. Using state-of-the-art LLMs, we extract and construct knowledge graphs and evaluate them comprehensively from both structural and semantic perspectives. Our results highlight the strengths and shortcomings of current LLMs in KG construction and identify key challenges for future work. To advance research in this area, we also release a curated dataset of LLM-generated KGs derived from research papers on children's mental well-being. This resource aims to foster more transparent, reliable, and impactful applications in high-stakes domains such as healthcare.
Abstract (translated)
知识图谱（KG）对于知识密集型任务至关重要，并且在减少大型语言模型（LLM）中的幻觉方面展现出潜力。然而，构建高质量的知识图谱仍然具有挑战性，需要准确的信息提取和能够支持可解释性和下游应用的结构化表示。现有的基于LLM的方法通常仅专注于实体和关系抽取，覆盖范围局限于句子级别的上下文或依赖于预定义的模式。我们提出了一种分层抽取框架，该框架在多个层次上组织信息，以创建语义丰富且结构良好的知识图谱。利用最先进的LLM，我们提取并构建了知识图谱，并从结构和语义两个方面对其进行了全面评估。我们的研究结果突出了当前LLM在知识图谱构建中的优势与不足，并确定了未来工作的关键挑战。 为了推动该领域的研究，我们还发布了一个精心策划的数据集，其中包含来自儿童心理健康研究论文的LLM生成的知识图谱。这一资源旨在促进医疗等高风险领域中更透明、可靠和有影响力的应用。
URL
https://arxiv.org/abs/2510.11297
PDF
-
LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corpora
2025-10-11 08:43:45Luyao Zhuang, Shengyuan Chen, Yilin Xiao, Huachi Zhou, Yujing Zhang, Hao Chen, Qinggang Zhang, Xiao HuangAbstract
Retrieval-Augmented Generation (RAG) is widely used to mitigate hallucinations of Large Language Models (LLMs) by leveraging external knowledge. While effective for simple queries, traditional RAG systems struggle with large-scale, unstructured corpora where information is fragmented. Recent advances incorporate knowledge graphs to capture relational structures, enabling more comprehensive retrieval for complex, multi-hop reasoning tasks. However, existing graph-based RAG (GraphRAG) methods rely on unstable and costly relation extraction for graph construction, often producing noisy graphs with incorrect or inconsistent relations that degrade retrieval quality. In this paper, we revisit the pipeline of existing GraphRAG systems and propose LinearRAG (Linear Graph-based Retrieval-Augmented Generation), an efficient framework that enables reliable graph construction and precise passage retrieval. Specifically, LinearRAG constructs a relation-free hierarchical graph, termed Tri-Graph, using only lightweight entity extraction and semantic linking, avoiding unstable relation modeling. This new paradigm of graph construction scales linearly with corpus size and incurs no extra token consumption, providing an economical and reliable indexing of the original passages. For retrieval, LinearRAG adopts a two-stage strategy: (i) relevant entity activation via local semantic bridging, followed by (ii) passage retrieval through global importance aggregation. Extensive experiments on four datasets demonstrate that LinearRAG significantly outperforms baseline models.
Abstract (translated)
检索增强生成（RAG）广泛用于通过利用外部知识来减少大规模语言模型（LLMs）的幻觉问题。虽然对于简单的查询非常有效，但传统的RAG系统在处理大型、无结构化且信息碎片化的语料库时会遇到困难。最近的研究进展中，知识图谱被用来捕捉关系结构，从而能够为复杂的多跳推理任务提供更全面的信息检索。然而，现有的基于图的RAG（GraphRAG）方法依赖于不稳定的和昂贵的关系提取来进行图构建，通常会产生含有错误或不一致关系的嘈杂图，这会降低检索质量。 在本文中，我们重新审视了现有GraphRAG系统的流程，并提出了LinearRAG（基于线性图的检索增强生成），这是一种高效的框架，能够实现可靠的图构造和精确的信息段检索。具体来说，LinearRAG使用轻量级实体提取和语义链接来构建无关系依赖的层次化图，即Tri-Graph，从而避免了不稳定的模型关系建模。这种新的图形构建范式与原始文档集的规模呈线性增长，并且不会产生额外的标记消耗，提供了一种经济而可靠的索引方式。 在检索方面，LinearRAG采用两阶段策略：首先是通过局部语义桥接激活相关实体，其次是通过全局重要性聚合进行信息段检索。在四个数据集上的广泛实验表明，LinearRAG显著优于基线模型。
URL
https://arxiv.org/abs/2510.10114
PDF
-
Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction
2025-09-25 13:38:38Lei Hei, Tingjing Liao, Yingxin Pei, Yiyang Qi, Jiaqi Wang, Ruiting Li, Feiliang RenAbstract
Relation extraction (RE) aims to identify semantic relations between entities in unstructured text. Although recent work extends traditional RE to multimodal scenarios, most approaches still adopt classification-based paradigms with fused multimodal features, representing relations as discrete labels. This paradigm has two significant limitations: (1) it overlooks structural constraints like entity types and positional cues, and (2) it lacks semantic expressiveness for fine-grained relation understanding. We propose \underline{R}etrieval \underline{O}ver \underline{C}lassification (ROC), a novel framework that reformulates multimodal RE as a retrieval task driven by relation semantics. ROC integrates entity type and positional information through a multimodal encoder, expands relation labels into natural language descriptions using a large language model, and aligns entity-relation pairs via semantic similarity-based contrastive learning. Experiments show that our method achieves state-of-the-art performance on the benchmark datasets MNRE and MORE and exhibits stronger robustness and interpretability.
Abstract (translated)
关系抽取（RE）旨在识别无结构文本中实体之间的语义关系。尽管最近的工作将传统的关系抽取扩展到了多模态场景，但大多数方法仍然采用基于分类的范式，并使用融合的多模态特征表示离散标签形式的关系。这种范式存在两个主要限制：(1) 忽视了如实体类型和位置提示等结构约束；(2) 在细粒度关系理解方面缺乏语义表达能力。我们提出了“检索超越分类”（ROC），这是一种新的框架，将多模态RE重新表述为一种由关系语义驱动的检索任务。ROC通过一个多模态编码器整合实体类型和位置信息，并利用大型语言模型将关系标签扩展为自然语言描述，同时通过基于语义相似性的对比学习来对齐实体-关系对。实验表明，我们的方法在基准数据集MNRE和MORE上实现了最先进的性能，并且表现出更强的鲁棒性和可解释性。
URL
https://arxiv.org/abs/2509.21151
PDF
-
Automated Knowledge Graph Construction using Large Language Models and Sentence Complexity Modelling
2025-09-22 00:01:50Sydney Anuyah, Mehedi Mahmud Kaushik, Krishna Dwarampudi, Rakesh Shiradkar, Arjan Durresi, Sunandan ChakrabortyAbstract
We introduce CoDe-KG, an open-source, end-to-end pipeline for extracting sentence-level knowledge graphs by combining robust coreference resolution with syntactic sentence decomposition. Using our model, we contribute a dataset of over 150,000 knowledge triples, which is open source. We also contribute a training corpus of 7248 rows for sentence complexity, 190 rows of gold human annotations for co-reference resolution using open source lung-cancer abstracts from PubMed, 900 rows of gold human annotations for sentence conversion policies, and 398 triples of gold human annotations. We systematically select optimal prompt-model pairs across five complexity categories, showing that hybrid chain-of-thought and few-shot prompting yields up to 99.8% exact-match accuracy on sentence simplification. On relation extraction (RE), our pipeline achieves 65.8% macro-F1 on REBEL, an 8-point gain over the prior state of the art, and 75.7% micro-F1 on WebNLG2, while matching or exceeding performance on Wiki-NRE and CaRB. Ablation studies demonstrate that integrating coreference and decomposition increases recall on rare relations by over 20%. Code and dataset are available at this https URL
Abstract (translated)
我们介绍了CoDe-KG，这是一个开源的、端到端的流水线，用于结合稳健的共指消解与句法句子分解来提取句子级别的知识图谱。使用我们的模型，我们贡献了一个包含超过150,000条知识三元组的数据集，并且该数据集是开放源代码的。我们也提供了7248行关于句子复杂度的训练语料库、用PubMed上的开源肺癌摘要进行共指消解的人工标注黄金标准注释190行，转换策略的人工标注黄金标准注释900行以及398条人工标注的黄金标准三元组。我们系统地选择五个复杂性类别中的最优提示-模型对，结果表明混合链式思维和少量样本提示可以将句子简化的确切匹配准确率提高到最高达99.8%。 在关系抽取（RE）方面，我们的流水线在REBEL上达到了65.8%的宏观F1分数，比之前的最佳性能提高了8个百分点，并且在WebNLG2上的微平均F1得分为75.7%，同时在Wiki-NRE和CaRB数据集上的表现也与最佳水平持平或超越。消融实验表明，将共指消解和句子分解集成后，在罕见关系的召回率上可以提高超过20%。 代码和数据集可以在[提供的链接]中获取。
URL
https://arxiv.org/abs/2509.17289
PDF
-
HARE: an entity and relation centric evaluation framework for histopathology reports
2025-09-19 18:12:19Yunsoo Kim, Michal W. S. Ong, Alex Shavick, Honghan Wu, Adam P. LevineAbstract
Medical domain automated text generation is an active area of research and development; however, evaluating the clinical quality of generated reports remains a challenge, especially in instances where domain-specific metrics are lacking, e.g. histopathology. We propose HARE (Histopathology Automated Report Evaluation), a novel entity and relation centric framework, composed of a benchmark dataset, a named entity recognition (NER) model, a relation extraction (RE) model, and a novel metric, which prioritizes clinically relevant content by aligning critical histopathology entities and relations between reference and generated reports. To develop the HARE benchmark, we annotated 813 de-identified clinical diagnostic histopathology reports and 652 histopathology reports from The Cancer Genome Atlas (TCGA) with domain-specific entities and relations. We fine-tuned GatorTronS, a domain-adapted language model to develop HARE-NER and HARE-RE which achieved the highest overall F1-score (0.915) among the tested models. The proposed HARE metric outperformed traditional metrics including ROUGE and Meteor, as well as radiology metrics such as RadGraph-XL, with the highest correlation and the best regression to expert evaluations (higher than the second best method, GREEN, a large language model based radiology report evaluator, by Pearson $r = 0.168$, Spearman $\rho = 0.161$, Kendall $\tau = 0.123$, $R^2 = 0.176$, $RMSE = 0.018$). We release HARE, datasets, and the models at this https URL to foster advancements in histopathology report generation, providing a robust framework for improving the quality of reports.
Abstract (translated)
医学领域的自动化文本生成是一个活跃的研究和开发领域；然而，评估所生成报告的临床质量仍然是一个挑战，尤其是在缺乏特定领域指标的情况下，例如组织病理学。我们提出了HARE（Histopathology Automated Report Evaluation），这是一种新颖以实体和关系为中心的框架，包括基准数据集、命名实体识别（NER）模型、关系抽取（RE）模型以及一种新的度量标准，该标准通过在参考报告与生成报告之间对关键组织病理学实体和关系进行对齐来优先考虑临床相关内容。为了开发HARE基准，我们为813份去身份化的临床诊断组织病理学报告和来自癌症基因组图谱（TCGA）的652份组织病理学报告进行了特定领域的实体和关系标注。我们将GatorTronS（一种领域适应的语言模型）微调以开发出具有最高整体F1分数（0.915，这是在测试的所有模型中最高的）的HARE-NER和HARE-RE。我们提出的HARE度量标准优于传统的包括ROUGE和Meteor在内的度量标准，以及放射学指标例如RadGraph-XL，在相关性、回归到专家评估方面表现出最高水平（高于第二种最佳方法GREEN，一种大型语言模型驱动的放射报告评估器，通过Pearson $r = 0.168$，Spearman $\rho = 0.161$，Kendall $\tau = 0.123$，$R^2 = 0.176$，$RMSE = 0.018$）。我们已经在[此处链接]发布HARE、数据集和模型，以促进组织病理学报告生成的改进，并为提高报告质量提供了强大的框架。
URL
https://arxiv.org/abs/2509.16326
PDF
-
SciEvent: Benchmarking Multi-domain Scientific Event Extraction
2025-09-19 05:32:50Bofu Dong, Pritesh Shah, Sumedh Sonawane, Tiyasha Banerjee, Erin Brady, Xinya Du, Ming JiangAbstract
Scientific information extraction (SciIE) has primarily relied on entity-relation extraction in narrow domains, limiting its applicability to interdisciplinary research and struggling to capture the necessary context of scientific information, often resulting in fragmented or conflicting statements. In this paper, we introduce SciEvent, a novel multi-domain benchmark of scientific abstracts annotated via a unified event extraction (EE) schema designed to enable structured and context-aware understanding of scientific content. It includes 500 abstracts across five research domains, with manual annotations of event segments, triggers, and fine-grained arguments. We define SciIE as a multi-stage EE pipeline: (1) segmenting abstracts into core scientific activities--Background, Method, Result, and Conclusion; and (2) extracting the corresponding triggers and arguments. Experiments with fine-tuned EE models, large language models (LLMs), and human annotators reveal a performance gap, with current models struggling in domains such as sociology and humanities. SciEvent serves as a challenging benchmark and a step toward generalizable, multi-domain SciIE.
Abstract (translated)
科学信息抽取（SciIE）主要依赖于狭窄领域的实体关系提取，这限制了其在跨学科研究中的适用性，并且难以捕捉到科学信息所需的背景语境，常常导致支离破碎或相互矛盾的表述。本文中，我们介绍了SciEvent，这是一个新的多领域基准测试集，通过统一的事件抽取（EE）模式对科学摘要进行了标注，旨在实现对科学内容的结构化和上下文感知的理解。该数据集包括了来自五个研究领域的500篇摘要，并对其事件片段、触发词及细粒度论点进行了手动注释。我们定义SciIE为一个多阶段的EE流水线：（1）将摘要分割为核心科学活动，如背景、方法、结果和结论；以及（2）提取相应的触发词和论点。 实验结果显示了通过微调事件抽取模型、大型语言模型（LLMs）及人工标注者之间的性能差距，在诸如社会学和人文学科等特定领域，当前的模型表现较差。SciEvent作为一个具有挑战性的基准测试集，并朝着通用性、多领域的SciIE迈进了一步。
URL
https://arxiv.org/abs/2509.15620
PDF
