Abstract
In the era of large language model, relation extraction (RE) plays an important role in information extraction through the transformation of unstructured raw text into structured data (Wadhwa et al., 2023). In this paper, we systematically compare the performance of deep supervised learning approaches without transformers and those with transformers. We used a series of non-transformer architectures such as PA-LSTM(Zhang et al., 2017), C-GCN(Zhang et al., 2018), and AGGCN(attention guide GCN)(Guo et al., 2019), and a series of transformer architectures such as BERT, RoBERTa, and R-BERT(Wu and He, 2019). Our comparison included traditional metrics like micro F1, as well as evaluations in different scenarios, varying sentence lengths, and different percentages of the dataset for training. Our experiments were conducted on TACRED, TACREV, and RE-TACRED. The results show that transformer-based models outperform non-transformer models, achieving micro F1 scores of 80-90% compared to 64-67% for non-transformer models. Additionally, we briefly review the research journey in supervised relation classification and discuss the role and current status of large language models (LLMs) in relation extraction.
Abstract (translated)
在大型语言模型时代，关系抽取（RE）通过将非结构化的原始文本转换为结构化数据，在信息提取中扮演着重要角色（Wadhwa等人，2023）。本文系统地比较了没有使用变换器的深度监督学习方法与使用变换器的方法之间的性能。我们采用了一系列非变换器架构，如PA-LSTM(Zhang等人，2017)，C-GCN(Zhang等人，2018)和AGGCN（注意力引导GCN）(Guo等人，2019)，以及一系列基于变换器的架构，例如BERT、RoBERTa和R-BERT(Wu和He, 2019)。我们的比较包括传统的指标如微平均F1分数，以及不同场景下的评估、不同的句子长度和数据集中用于训练的不同百分比的数据集。我们在TACRED、TACREV和RE-TACRED上进行了实验。结果显示，基于变换器的模型优于非变换器模型，在微平均F1分数方面达到了80%-90%，而后者仅为64%-67%。此外，我们还简要回顾了监督关系分类的研究历程，并讨论了大规模语言模型（LLMs）在关系抽取中的作用和当前地位。
URL
https://arxiv.org/abs/2509.11374
