Abstract
Vision Transformers (ViTs) have demonstrated impressive performance across a wide range of biometric tasks, including face and body recognition. In this work, we adapt a ViT model pretrained on visible (VIS) imagery to the challenging problem of cross-spectral body recognition, which involves matching images captured in the visible and infrared (IR) domains. Recent ViT architectures have explored incorporating additional embeddings beyond traditional positional embeddings. Building on this idea, we integrate Side Information Embedding (SIE) and examine the impact of encoding domain and camera information to enhance cross-spectral matching. Surprisingly, our results show that encoding only camera information - without explicitly incorporating domain information - achieves state-of-the-art performance on the LLCM dataset. While occlusion handling has been extensively studied in visible-spectrum person re-identification (Re-ID), occlusions in visible-infrared (VI) Re-ID remain largely underexplored - primarily because existing VI-ReID datasets, such as LLCM, SYSU-MM01, and RegDB, predominantly feature full-body, unoccluded images. To address this gap, we analyze the impact of range-induced occlusions using the IARPA Janus Benchmark Multi-Domain Face (IJB-MDF) dataset, which provides a diverse set of visible and infrared images captured at various distances, enabling cross-range, cross-spectral evaluations.
Abstract (translated)
视觉变压器（ViT）在包括面部和身体识别在内的多种生物特征任务中表现出色。在这项工作中，我们将一个预训练于可见光图像的ViT模型应用于跨谱系身体识别这一具有挑战性的问题上，该问题涉及到匹配可视域和红外域捕获的图像。最近的ViT架构探索了在传统位置嵌入之外添加额外嵌入的方法。在此基础上，我们整合了侧信息嵌入（SIE），并研究编码领域和相机信息对跨谱系匹配的影响。令人惊讶的是，我们的结果显示，在LLCM数据集上，仅通过编码相机信息——而无需明确地将领域信息纳入考虑内——即可达到最先进的性能。 尽管在可见光谱范围内的人员重新识别（Re-ID）中已经广泛研究了遮挡处理方法，但针对可见-红外（VI）Re-ID中的遮挡问题尚未得到充分探索。这主要是因为现有的VI-ReID数据集（如LLCM、SYSU-MM01和RegDB），主要包含的是全身且未被遮挡的图像。 为了解决这一差距，我们利用IARPA Janus基准多域面部识别（IJB-MDF）数据集分析了由范围引起的遮挡影响。该数据集提供了一组多样化的可见光和红外图像，在不同距离下拍摄，支持跨范围、跨谱系评估。
URL
https://arxiv.org/abs/2506.08953
