Abstract
Biometric recognition becomes increasingly challenging as we move away from the visible spectrum to infrared imagery, where domain discrepancies significantly impact identification performance. In this paper, we show that body embeddings perform better than face embeddings for cross-spectral person identification in medium-wave infrared (MWIR) and long-wave infrared (LWIR) domains. Due to the lack of multi-domain datasets, previous research on cross-spectral body identification - also known as Visible-Infrared Person Re-Identification (VI-ReID) - has primarily focused on individual infrared bands, such as near-infrared (NIR) or LWIR, separately. We address the multi-domain body recognition problem using the IARPA Janus Benchmark Multi-Domain Face (IJB-MDF) dataset, which enables matching of short-wave infrared (SWIR), MWIR, and LWIR images against RGB (VIS) images. We leverage a vision transformer architecture to establish benchmark results on the IJB-MDF dataset and, through extensive experiments, provide valuable insights into the interrelation of infrared domains, the adaptability of VIS-pretrained models, the role of local semantic features in body-embeddings, and effective training strategies for small datasets. Additionally, we show that finetuning a body model, pretrained exclusively on VIS data, with a simple combination of cross-entropy and triplet losses achieves state-of-the-art mAP scores on the LLCM dataset.
Abstract (translated)
随着我们从可见光谱过渡到红外图像，生物识别技术面临的挑战日益增加，因为不同领域的差异显著影响了识别性能。本文展示了在中波红外（MWIR）和长波红外（LWIR）领域，人体嵌入式模型比面部嵌入式模型更适用于跨光谱的人体识别。 由于缺乏多域数据集，之前关于跨光谱人体识别的研究——即可见光-红外人再识别（VI-ReID），主要集中在近红外（NIR）或长波红外等单独的红外频段上。本文使用IARPA Janus Benchmark Multi-Domain Face (IJB-MDF) 数据集解决了多域人体识别问题，该数据集支持将短波红外(SWIR)，MWIR 和 LWIR 图像与RGB图像匹配。 我们利用视觉变换器架构在 IJB-MDF 数据集上建立了基准结果，并通过广泛的实验提供了关于红外频段之间相互关系的宝贵见解、可见光预训练模型的适应性、人体嵌入中的局部语义特征的作用，以及小数据集中有效的训练策略。此外，本文还展示了仅在VIS数据上进行预训练的人体模型，在使用简单的交叉熵和三元组损失组合微调后，在LLCM 数据集上达到了最先进的平均精度（mAP）分数。
URL
https://arxiv.org/abs/2503.10931
