Abstract
Visible-Infrared Person Re-identification (VIReID) aims to match visible and infrared pedestrian images, but the modality differences and the complexity of identity features make it challenging. Existing methods rely solely on identity label supervision, which makes it difficult to fully extract high-level semantic information. Recently, vision-language pre-trained models have been introduced to VIReID, enhancing semantic information modeling by generating textual descriptions. However, such methods do not explicitly model body shape features, which are crucial for cross-modal matching. To address this, we propose an effective Body Shape-aware Textual Alignment (BSaTa) framework that explicitly models and utilizes body shape information to improve VIReID performance. Specifically, we design a Body Shape Textual Alignment (BSTA) module that extracts body shape information using a human parsing model and converts it into structured text representations via CLIP. We also design a Text-Visual Consistency Regularizer (TVCR) to ensure alignment between body shape textual representations and visual body shape features. Furthermore, we introduce a Shape-aware Representation Learning (SRL) mechanism that combines Multi-text Supervision and Distribution Consistency Constraints to guide the visual encoder to learn modality-invariant and discriminative identity features, thus enhancing modality invariance. Experimental results demonstrate that our method achieves superior performance on the SYSU-MM01 and RegDB datasets, validating its effectiveness.
Abstract (translated)
可见光-红外行人重识别（VIReID）旨在匹配可见光和红外图像中的行人，但由于模态差异以及身份特征的复杂性，这使得任务具有挑战性。现有的方法主要依赖于身份标签监督来提取高层语义信息，这种方法难以全面利用高阶语义信息。最近，视觉-语言预训练模型被引入到VIReID中，通过生成文本描述来增强语义信息建模。然而，这些方法没有显式地对体形特征进行建模，而这对于跨模态匹配至关重要。 为了解决这个问题，我们提出了一种有效的基于体形的文本对齐（Body Shape-aware Textual Alignment, BSaTa）框架，该框架通过利用和建模体形信息来提高VIReID的表现。具体来说，我们设计了一个体形文本对齐模块（BSTA），它使用人体解析模型提取体形特征，并通过CLIP将其转换为结构化文本表示形式。此外，我们还设计了一种基于文本-视觉一致性的正则器（Text-Visual Consistency Regularizer, TVCR）来确保体形文本表示与视觉中的体形特征对齐。 为了进一步提升表现，我们引入了形状感知表征学习机制（Shape-aware Representation Learning, SRL），该机制结合了多文本监督和分布一致性约束，指导视觉编码器学习模态不变且具有判别性的身份特征。实验结果表明，在SYSU-MM01和RegDB数据集上，我们的方法取得了优越的表现，验证了其有效性。 这种新的框架不仅增强了语义信息建模，还通过显式地考虑体形特征来提高了跨模态匹配的准确性，并在一定程度上解决了不同模态之间的不一致性问题。
URL
https://arxiv.org/abs/2504.18025
