Abstract
Relation extraction (RE) aims to identify semantic relations between entities in unstructured text. Although recent work extends traditional RE to multimodal scenarios, most approaches still adopt classification-based paradigms with fused multimodal features, representing relations as discrete labels. This paradigm has two significant limitations: (1) it overlooks structural constraints like entity types and positional cues, and (2) it lacks semantic expressiveness for fine-grained relation understanding. We propose \underline{R}etrieval \underline{O}ver \underline{C}lassification (ROC), a novel framework that reformulates multimodal RE as a retrieval task driven by relation semantics. ROC integrates entity type and positional information through a multimodal encoder, expands relation labels into natural language descriptions using a large language model, and aligns entity-relation pairs via semantic similarity-based contrastive learning. Experiments show that our method achieves state-of-the-art performance on the benchmark datasets MNRE and MORE and exhibits stronger robustness and interpretability.
Abstract (translated)
关系抽取（RE）旨在识别无结构文本中实体之间的语义关系。尽管最近的工作将传统的关系抽取扩展到了多模态场景，但大多数方法仍然采用基于分类的范式，并使用融合的多模态特征表示离散标签形式的关系。这种范式存在两个主要限制：(1) 忽视了如实体类型和位置提示等结构约束；(2) 在细粒度关系理解方面缺乏语义表达能力。我们提出了“检索超越分类”（ROC），这是一种新的框架，将多模态RE重新表述为一种由关系语义驱动的检索任务。ROC通过一个多模态编码器整合实体类型和位置信息，并利用大型语言模型将关系标签扩展为自然语言描述，同时通过基于语义相似性的对比学习来对齐实体-关系对。实验表明，我们的方法在基准数据集MNRE和MORE上实现了最先进的性能，并且表现出更强的鲁棒性和可解释性。
URL
https://arxiv.org/abs/2509.21151
