Abstract
Large, high-quality annotated corpora remain scarce in document-level entity and relation extraction in zero-shot or few-shot settings. In this paper, we present a fully automatic, LLM-based pipeline for synthetic data generation and in-context learning for document-level entity and relation extraction. In contrast to existing approaches that rely on manually annotated demonstrations or direct zero-shot inference, our method combines synthetic data generation with retrieval-based in-context learning, using a reasoning-optimized language model. This allows us to build a high-quality demonstration database without manual annotation and to dynamically retrieve relevant examples at inference time. Based on our approach we produce a synthetic dataset of over $5k$ Wikipedia abstracts with approximately $59k$ entities and $30k$ relation triples. Finally, we evaluate in-context learning performance on the DocIE shared task, extracting entities and relations from long documents in a zero-shot setting. We find that in-context joint entity and relation extraction at document-level remains a challenging task, even for state-of-the-art large language models.
Abstract (translated)
在文档级别的实体和关系抽取中，特别是在零样本或小样本设置下，大型高质量的标注语料库仍然非常稀缺。本文提出了一种基于大语言模型（LLM）的全自动数据生成及上下文学习管道，用于合成数据生成以及文档级实体与关系抽取任务中的零样本推理。 相较于现有方法依赖于手动注释示例或直接进行零样本推理，我们的方法结合了合成数据生成和检索增强型上下文学习技术，并使用经过优化以提高推理能力的语言模型。这样，我们能够在不需人工标注的情况下构建高质量的示范数据库，并在推断时动态地检索相关实例。 基于这一方法，我们生成了一个包含超过5000篇维基百科摘要的合成数据集，其中约有59,000个实体和30,000条关系三元组。最后，我们在DocIE共享任务上评估了上下文学习在零样本设置下从长文档中抽取实体和关系的表现。 我们发现，即使对于最先进的大型语言模型而言，在文档级别进行零样本联合实体与关系抽取仍然是一个具有挑战性的任务。
URL
https://arxiv.org/abs/2507.05997
