Abstract
Visible-Infrared Person Re-Identification (VI-ReID) is a challenging task due to the large modality discrepancy between visible and infrared images, which complicates the alignment of their features into a suitable common space. Moreover, style noise, such as illumination and color contrast, reduces the identity discriminability and modality invariance of features. To address these challenges, we propose a novel Diverse Semantics-guided Feature Alignment and Decoupling (DSFAD) network to align identity-relevant features from different modalities into a textual embedding space and disentangle identity-irrelevant features within each modality. Specifically, we develop a Diverse Semantics-guided Feature Alignment (DSFA) module, which generates pedestrian descriptions with diverse sentence structures to guide the cross-modality alignment of visual features. Furthermore, to filter out style information, we propose a Semantic Margin-guided Feature Decoupling (SMFD) module, which decomposes visual features into pedestrian-related and style-related components, and then constrains the similarity between the former and the textual embeddings to be at least a margin higher than that between the latter and the textual embeddings. Additionally, to prevent the loss of pedestrian semantics during feature decoupling, we design a Semantic Consistency-guided Feature Restitution (SCFR) module, which further excavates useful information for identification from the style-related features and restores it back into the pedestrian-related features, and then constrains the similarity between the features after restitution and the textual embeddings to be consistent with that between the features before decoupling and the textual embeddings. Extensive experiments on three VI-ReID datasets demonstrate the superiority of our DSFAD.
Abstract (translated)
可见光-红外重识别（VI-ReID）是一项具有挑战性的任务，由于可见光和红外图像之间的模态差异较大，使得将它们的特征对齐到一个合适的公共空间变得复杂。此外，如光照和色彩对比等风格噪声降低了身份判别性和跨模态不变性。为了解决这些挑战，我们提出了一种新颖的基于多样化语义指导的特征对齐与解耦（DSFAD）网络，旨在将不同模态中的身份相关特征对齐到文本嵌入空间，并在每种模态内部分离出身份无关的特征。 具体而言，我们开发了一个基于多样化语义引导的特征对齐模块（DSFA），该模块生成具有多样句式结构的人行道描述以指导视觉特征之间的跨模态对齐。此外，为了过滤掉风格信息，我们提出了一种基于语义间隔导向的特征解耦模块（SMFD），将视觉特征分解为人行道相关和风格相关的部分，并且限制前者与文本嵌入相似度至少高于后者与文本嵌入的相似度一定阈值之上。另外，为防止在特征解耦过程中丢失行人语义信息，我们设计了一种基于语义一致性导向的特征恢复模块（SCFR），该模块进一步从风格相关特征中挖掘出识别有用的信息并将其恢复到人行道相关特征中，并且限制了经过恢复后与文本嵌入相似性的保持一致性。 在三个VI-ReID数据集上的大量实验结果证明了我们DSFAD网络的优越性。
URL
https://arxiv.org/abs/2505.00619
