Abstract
Visible-infrared person re-identification (VI-ReID) aims to match individuals across different camera modalities, a critical task in modern surveillance systems. While current VI-ReID methods focus on cross-modality matching, real-world applications often involve mixed galleries containing both V and I images, where state-of-the-art methods show significant performance limitations due to large domain shifts and low discrimination across mixed modalities. This is because gallery images from the same modality may have lower domain gaps but correspond to different identities. This paper introduces a novel mixed-modal ReID setting, where galleries contain data from both modalities. To address the domain shift among inter-modal and low discrimination capacity in intra-modal matching, we propose the Mixed Modality-Erased and -Related (MixER) method. The MixER learning approach disentangles modality-specific and modality-shared identity information through orthogonal decomposition, modality-confusion, and ID-modality-related objectives. MixER enhances feature robustness across modalities, improving cross-modal and mixed-modal settings performance. Our extensive experiments on the SYSU-MM01, RegDB and LLMC datasets indicate that our approach can provide state-of-the-art results using a single backbone, and showcase the flexibility of our approach in mixed gallery applications.
Abstract (translated)
可见光-红外人物再识别（VI-ReID）旨在跨不同相机模态匹配个体，这是现代监控系统中的一个关键任务。尽管当前的VI-ReID方法专注于跨模态匹配，但在实际应用中，混合图像库通常包含可见光(V)和红外(I)两种图像的情况很常见。对于这种场景，最先进的方法在处理大规模领域偏移和低跨模态识别能力时表现出明显的性能限制。这是因为来自同一模态的图库图像可能具有较小的领域差距但对应于不同的身份。 本文提出了一种新的混合模式下的ReID设置，在该设置中图库包含两种模态的数据。为了解决跨模态之间的领域偏移和低模内匹配识别能力的问题，我们提出了混合模式擦除与相关（MixER）方法。通过正交分解、模态混淆以及身份-模态相关的目标，MixER学习方法能够分离出特定于模态的和个人共享的身份信息。 MixER增强了跨模态特征的鲁棒性，从而提高了跨模和混合设置下的性能表现。我们在SYSU-MM01、RegDB及LLMC数据集上的广泛实验表明，我们的方法可以使用单一主干网络提供最先进的结果，并展示了在混合图库应用中的灵活性和适用性。
URL
https://arxiv.org/abs/2501.13307
