Abstract
Multi-modal vehicle Re-Identification (ReID) aims to leverage complementary information from RGB, Near Infrared (NIR), and Thermal Infrared (TIR) modalities to retrieve the same vehicle. The challenges of multi-modal vehicle ReID arise from the uncertainty of modality quality distribution induced by inherent discrepancies across modalities, resulting in distinct conflicting fusion requirements for data with balanced and unbalanced quality distributions. Existing methods handle all multi-modal data within a single fusion model, overlooking the different needs of the two data types and making it difficult to decouple the conflict between intra-class consistency and inter-modal heterogeneity. To this end, we propose Disentangle Collaboration and Guidance Fusion Representations for Multi-modal Vehicle ReID (DCG-ReID). Specifically, to disentangle heterogeneous quality-distributed modal data without mutual interference, we first design the Dynamic Confidence-based Disentangling Weighting (DCDW) mechanism: dynamically reweighting three-modal contributions via interaction-derived modal confidence to build a disentangled fusion framework. Building on DCDW, we develop two scenario-specific fusion strategies: (1) for balanced quality distributions, Collaboration Fusion Module (CFM) mines pairwise consensus features to capture shared discriminative information and boost intra-class consistency; (2) for unbalanced distributions, Guidance Fusion Module (GFM) implements differential amplification of modal discriminative disparities to reinforce dominant modality advantages, guide auxiliary modalities to mine complementary discriminative info, and mitigate inter-modal divergence to boost multi-modal joint decision performance. Extensive experiments on three multi-modal ReID benchmarks (WMVeID863, MSVR310, RGBNT100) validate the effectiveness of our method. Code will be released upon acceptance.
Abstract (translated)
多模态车辆再识别（ReID）旨在利用RGB、近红外（NIR）和热红外（TIR）等不同模式提供的互补信息，来检索同一辆车。这一任务的挑战在于，由于各模式之间的固有差异性导致了它们质量分布的不确定性，从而产生了对平衡和不平衡质量数据融合的不同要求。现有的方法通常采用单一的融合模型处理所有多模态数据，忽视了两种类型数据的具体需求，并且难以分离类内一致性与跨模态异质性的矛盾。 为了解决这些问题，我们提出了“分解协作和指导性融合表示用于多模态车辆ReID”（DCG-ReID）。具体来说，为了在不互相干扰的情况下将不同质量分布的异构模式数据解耦，我们首先设计了基于动态置信度的分离加权机制（DCDW）：通过由交互得出的模式自信度来动态地重新调整三模态贡献权重，并构建一个解耦的融合框架。在此基础上，针对两种不同的场景分别开发出两个特定的融合策略： 1. 对于平衡质量分布的情况，协作融合模块（CFM）挖掘成对共识特征以捕捉共享区分信息并增强类内一致性。 2. 对于不平衡分布情况，则采用指导性融合模块（GFM），通过差异化放大模式间的区分差异来加强主导模态的优势，并引导辅助模态去挖掘补充的区分信息，从而缓解跨模态分歧，提升多模态联合决策性能。 我们已经在三个多模态ReID基准测试集（WMVeID863、MSVR310和RGBNT100）上进行了广泛的实验验证了该方法的有效性。在论文被接受后将公开代码。
URL
https://arxiv.org/abs/2601.02924
