Abstract
Continual Relation Extraction (CRE) aims to continually learn new emerging relations while avoiding catastrophic forgetting. Existing CRE methods mainly use memory replay and contrastive learning to mitigate catastrophic forgetting. However, these methods do not attach importance to the error cases that can reveal the model's cognitive biases more effectively. To address this issue, we propose an instruction-based continual contrastive tuning approach for Large Language Models (LLMs) in CRE. Different from existing CRE methods that typically handle the training and memory data in a unified manner, this approach splits the training and memory data of each task into two parts respectively based on the correctness of the initial responses and treats them differently through dual-task fine-tuning. In addition, leveraging the advantages of LLM's instruction-following ability, we propose a novel instruction-based contrastive tuning strategy for LLM to continuously correct current cognitive biases with the guidance of previous data in an instruction-tuning manner, which mitigates the gap between old and new relations in a more suitable way for LLMs. We experimentally evaluate our model on TACRED and FewRel, and the results show that our model achieves new state-of-the-art CRE performance with significant improvements, demonstrating the importance of specializing in exploiting error cases.
Abstract (translated)
持续关系抽取（CRE）的目标是不断学习新出现的关系，同时避免灾难性遗忘。现有的CRE方法主要采用记忆回放和对比学习来减轻灾难性遗忘问题。然而，这些方法并未充分重视能够更有效地揭示模型认知偏差的错误案例。为解决这一问题，我们提出了一种基于指令的持续对比微调方法，适用于大型语言模型（LLMs）在CRE中的应用。与现有的CRE方法通常以统一方式处理训练和记忆数据不同，该方法根据初始响应的正确性将每个任务的训练数据和记忆数据分别划分为两部分，并通过双任务微调进行区别对待。此外，利用大型语言模型遵循指令的优势，我们提出了一种新颖的基于指令的对比微调策略，使LLM能够持续地在指令调优模式下用先前的数据指导纠正当前的认知偏差，以更符合LLMs特点的方式缓解旧关系和新关系之间的差距。我们在TACRED和FewRel数据集上实验验证了我们的模型，并且结果显示，与现有方法相比，我们的模型取得了新的最优CRE性能，证明了专注于利用错误案例的重要性。
URL
https://arxiv.org/abs/2508.12031
