Abstract
Composed Image Retrieval (CIR) is a challenging image retrieval paradigm. It aims to retrieve target images from large-scale image databases that are consistent with the modification semantics, based on a multimodal query composed of a reference image and modification text. Although existing methods have made significant progress in cross-modal alignment and feature fusion, a key flaw remains: the neglect of contextual information in discriminating matching samples. However, addressing this limitation is not an easy task due to two challenges: 1) implicit dependencies and 2) the lack of a differential amplification mechanism. To address these challenges, we propose a dual-patH composItional coNtextualized neTwork (HINT), which can perform contextualized encoding and amplify the similarity differences between matching and non-matching samples, thus improving the upper performance of CIR models in complex scenarios. Our HINT model achieves optimal performance on all metrics across two CIR benchmark datasets, demonstrating the superiority of our HINT model. Codes are available at this https URL.
Abstract (translated)
组合图像检索（CIR）是一种具有挑战性的图像检索范式，其目标是根据由参考图像和修改文本组成的多模态查询，从大规模图像数据库中检索与修改语义一致的目标图像。尽管现有方法在跨模态对齐与特征融合方面已取得显著进展，但仍存在一个关键缺陷：在判别匹配样本时忽略了上下文信息。然而，解决这一局限性并非易事，主要面临两大挑战：1）隐式依赖关系；2）缺乏差异放大机制。为应对这些挑战，我们提出了一种双路径组合上下文网络（HINT），该模型能够实现上下文化编码，并放大匹配样本与非匹配样本之间的相似度差异，从而提升CIR模型在复杂场景中的性能上限。我们的HINT模型在两个CIR基准数据集上的所有指标均达到最优性能，充分证明了其优越性。代码已公开于此链接。
URL
https://arxiv.org/abs/2603.26341
