Person_Re-identification
-
Embedding and Enriching Explicit Semantics for Visible-Infrared Person Re-Identification
2024-12-11 14:27:30Neng Dong, Shuanglin Yan, Liyan Zhang, Jinhui TangAbstract
Visible-infrared person re-identification (VIReID) retrieves pedestrian images with the same identity across different modalities. Existing methods learn visual content solely from images, lacking the capability to sense high-level semantics. In this paper, we propose an Embedding and Enriching Explicit Semantics (EEES) framework to learn semantically rich cross-modality pedestrian representations. Our method offers several contributions. First, with the collaboration of multiple large language-vision models, we develop Explicit Semantics Embedding (ESE), which automatically supplements language descriptions for pedestrians and aligns image-text pairs into a common space, thereby learning visual content associated with explicit semantics. Second, recognizing the complementarity of multi-view information, we present Cross-View Semantics Compensation (CVSC), which constructs multi-view image-text pair representations, establishes their many-to-many matching, and propagates knowledge to single-view representations, thus compensating visual content with its missing cross-view semantics. Third, to eliminate noisy semantics such as conflicting color attributes in different modalities, we design Cross-Modality Semantics Purification (CMSP), which constrains the distance between inter-modality image-text pair representations to be close to that between intra-modality image-text pair representations, further enhancing the modality-invariance of visual content. Finally, experimental results demonstrate the effectiveness and superiority of the proposed EEES.
Abstract (translated)
可见光-红外行人再识别（VIReID）在不同的模态间检索同一身份的行人群图像。现有的方法仅从图像中学习视觉内容，缺乏感知高级语义的能力。本文提出了一种嵌入和丰富显式语义（EEES）框架来学习语义丰富的跨模态行人表示。我们的方法提供了几方面的贡献。首先，在多个大型语言-视觉模型的协作下，我们开发了显式语义嵌入（ESE），该技术自动为行人群补充语言描述，并将图像-文本对对齐到一个共同的空间中，从而学习与显式语义相关的视觉内容。其次，认识到多视角信息的互补性，我们提出了跨视图语义补偿（CVSC），它构建了多视图图像-文本对表示形式，建立了它们之间的多对多匹配，并将知识传播给单视图表示，从而用缺失的跨视图语义来补充视觉内容。第三，为了消除不同模态中的冲突颜色属性等噪声语义，我们设计了跨模态语义净化（CMSP），该技术限制不同模态间图像-文本对表示之间的距离接近同一模态内图像-文本对表示间的距离，进一步增强了视觉内容的模态不变性。最后，实验结果证明了所提出的EEES的有效性和优越性。
URL
https://arxiv.org/abs/2412.08406
PDF
-
Dynamic Modality-Camera Invariant Clustering for Unsupervised Visible-Infrared Person Re-identification
2024-12-11 09:31:03Yiming Yang, Weipeng Hu, Haifeng HuAbstract
Unsupervised learning visible-infrared person re-identification (USL-VI-ReID) offers a more flexible and cost-effective alternative compared to supervised methods. This field has gained increasing attention due to its promising potential. Existing methods simply cluster modality-specific samples and employ strong association techniques to achieve instance-to-cluster or cluster-to-cluster cross-modality associations. However, they ignore cross-camera differences, leading to noticeable issues with excessive splitting of identities. Consequently, this undermines the accuracy and reliability of cross-modal associations. To address these issues, we propose a novel Dynamic Modality-Camera Invariant Clustering (DMIC) framework for USL-VI-ReID. Specifically, our DMIC naturally integrates Modality-Camera Invariant Expansion (MIE), Dynamic Neighborhood Clustering (DNC) and Hybrid Modality Contrastive Learning (HMCL) into a unified framework, which eliminates both the cross-modality and cross-camera discrepancies in clustering. MIE fuses inter-modal and inter-camera distance coding to bridge the gaps between modalities and cameras at the clustering level. DNC employs two dynamic search strategies to refine the network's optimization objective, transitioning from improving discriminability to enhancing cross-modal and cross-camera generalizability. Moreover, HMCL is designed to optimize instance-level and cluster-level distributions. Memories for intra-modality and inter-modality training are updated using randomly selected samples, facilitating real-time exploration of modality-invariant representations. Extensive experiments have demonstrated that our DMIC addresses the limitations present in current clustering approaches and achieve competitive performance, which significantly reduces the performance gap with supervised methods.
Abstract (translated)
无监督学习可见光-红外行人重识别（USL-VI-ReID）相比有监督方法提供了更灵活且成本更低的替代方案。由于其潜在的应用前景，这一领域受到了越来越多的关注。现有方法简单地对模态特定样本进行聚类，并采用强大的关联技术实现实例到集群或集群到集群的跨模态关联。然而，它们忽略了跨摄像机差异，导致身份过度分割的问题显著存在。这削弱了跨模态关联的准确性和可靠性。为了解决这些问题，我们提出了一种新的动态模态-摄像机不变聚类（DMIC）框架用于USL-VI-ReID。具体来说，我们的DMIC自然地将模态-摄像机不变扩展（MIE）、动态邻域聚类（DNC）和混合模态对比学习（HMCL）整合到一个统一的框架中，这消除了聚类中的跨模态和跨摄像机差异。MIE融合了跨模态和跨摄像机的距离编码，在聚类层面弥合了模态和摄像机之间的差距。DNC采用了两种动态搜索策略来优化网络的目标函数，从提高可区分性转变为增强跨模态和跨摄像机的泛化能力。此外，HMCL旨在优化实例级别和集群级别的分布。使用随机选择的样本更新模态内和跨模态训练的记忆，有助于实时探索模态不变表征。广泛的实验表明，我们的DMIC解决了当前聚类方法中存在的局限性，并实现了具有竞争力的表现，显著缩小了与监督方法之间的性能差距。
URL
https://arxiv.org/abs/2412.08231
PDF
-
Mix-Modality Person Re-Identification: A New and Practical Paradigm
2024-12-06 02:19:57Wei Liu, Xin Xu, Hua Chang, Xin Yuan, Zheng WangAbstract
Current visible-infrared cross-modality person re-identification research has only focused on exploring the bi-modality mutual retrieval paradigm, and we propose a new and more practical mix-modality retrieval paradigm. Existing Visible-Infrared person re-identification (VI-ReID) methods have achieved some results in the bi-modality mutual retrieval paradigm by learning the correspondence between visible and infrared modalities. However, significant performance degradation occurs due to the modality confusion problem when these methods are applied to the new mix-modality paradigm. Therefore, this paper proposes a Mix-Modality person re-identification (MM-ReID) task, explores the influence of modality mixing ratio on performance, and constructs mix-modality test sets for existing datasets according to the new mix-modality testing paradigm. To solve the modality confusion problem in MM-ReID, we propose a Cross-Identity Discrimination Harmonization Loss (CIDHL) adjusting the distribution of samples in the hyperspherical feature space, pulling the centers of samples with the same identity closer, and pushing away the centers of samples with different identities while aggregating samples with the same modality and the same identity. Furthermore, we propose a Modality Bridge Similarity Optimization Strategy (MBSOS) to optimize the cross-modality similarity between the query and queried samples with the help of the similar bridge sample in the gallery. Extensive experiments demonstrate that compared to the original performance of existing cross-modality methods on MM-ReID, the addition of our CIDHL and MBSOS demonstrates a general improvement.
Abstract (translated)
当前的可见光-红外跨模态行人再识别研究仅集中在探索双模态互检范式上，我们提出了一种新的、更实用的混合模态检索范式。现有的可见光-红外行人再识别（VI-ReID）方法通过学习可见光和红外模态之间的对应关系，在双模态互检范式中取得了一些成果。然而，当这些方法应用于新的混合模态范式时，由于模态混淆问题导致性能显著下降。因此，本文提出了一个混合模态行人再识别（MM-ReID）任务，探讨了模态混合比例对性能的影响，并根据新的混合模态测试范式为现有数据集构建了混合模态测试集。为了在MM-ReID中解决模态混淆问题，我们提出了一种跨身份判别和谐损失（CIDHL），调整超球特征空间中的样本分布，拉近具有相同身份的样本中心的距离，并将不同身份的样本中心推开，同时聚集同一模态和相同身份的样本。此外，我们还提出了一个模态桥相似性优化策略（MBSOS），利用图像库中的类似桥梁样本帮助优化查询与被查询样本之间的跨模态相似度。广泛的实验表明，与现有跨模态方法在MM-ReID上的原始性能相比，加入我们的CIDHL和MBSOS显示出了一般性的改进。
URL
https://arxiv.org/abs/2412.04719
PDF
-
See What You Seek: Semantic Contextual Integration for Cloth-Changing Person Re-Identification
2024-12-02 10:11:16Xiyu Han, Xian Zhong, Wenxin Huang, Xuemei Jia, Wenxuan Liu, Xiaohan Yu, Alex Chichung KotAbstract
Cloth-changing person re-identification (CC-ReID) aims to match individuals across multiple surveillance cameras despite variations in clothing. Existing methods typically focus on mitigating the effects of clothing changes or enhancing ID-relevant features but often struggle to capture complex semantic information. In this paper, we propose a novel prompt learning framework, Semantic Contextual Integration (SCI), for CC-ReID, which leverages the visual-text representation capabilities of CLIP to minimize the impact of clothing changes and enhance ID-relevant features. Specifically, we introduce Semantic Separation Enhancement (SSE) module, which uses dual learnable text tokens to separately capture confounding and clothing-related semantic information, effectively isolating ID-relevant features from distracting clothing semantics. Additionally, we develop a Semantic-Guided Interaction Module (SIM) that uses orthogonalized text features to guide visual representations, sharpening the model's focus on distinctive ID characteristics. This integration enhances the model's discriminative power and enriches the visual context with high-dimensional semantic insights. Extensive experiments on three CC-ReID datasets demonstrate that our method outperforms state-of-the-art techniques. The code will be released at github.
Abstract (translated)
URL
https://arxiv.org/abs/2412.01345
PDF
-
Cerberus: Attribute-based person re-identification using semantic IDs
2024-12-02 02:12:28Chanho Eom, Geon Lee, Kyunghwan Cho, Hyeonseok Jung, Moonsub Jin, Bumsub HamAbstract
We introduce a new framework, dubbed Cerberus, for attribute-based person re-identification (reID). Our approach leverages person attribute labels to learn local and global person representations that encode specific traits, such as gender and clothing style. To achieve this, we define semantic IDs (SIDs) by combining attribute labels, and use a semantic guidance loss to align the person representations with the prototypical features of corresponding SIDs, encouraging the representations to encode the relevant semantics. Simultaneously, we enforce the representations of the same person to be embedded closely, enabling recognizing subtle differences in appearance to discriminate persons sharing the same attribute labels. To increase the generalization ability on unseen data, we also propose a regularization method that takes advantage of the relationships between SID prototypes. Our framework performs individual comparisons of local and global person representations between query and gallery images for attribute-based reID. By exploiting the SID prototypes aligned with the corresponding representations, it can also perform person attribute recognition (PAR) and attribute-based person search (APS) without bells and whistles. Experimental results on standard benchmarks on attribute-based person reID, Market-1501 and DukeMTMC, demonstrate the superiority of our model compared to the state of the art.
Abstract (translated)
URL
https://arxiv.org/abs/2412.01048
PDF
-
Dynamic Token Selection for Aerial-Ground Person Re-Identification
2024-11-30 11:07:11Yuhai WangAbstract
We propose a View-Decoupled Transformer (VDT) framework to address viewpoint discrepancies in person re-identification (ReID), particularly between aerial and ground views. VDT decouples view-specific and view-independent features by leveraging meta and view tokens, processed through self-attention and subtractive separation. Additionally, we introduce a Visual Token Selector (VTS) module that dynamically selects the most informative tokens, reducing redundancy and enhancing efficiency. Our approach significantly improves retrieval performance on the AGPReID dataset, while maintaining computational efficiency similar to baseline models.
Abstract (translated)
URL
https://arxiv.org/abs/2412.00433
PDF
-
Cross-Spectral Attention for Unsupervised RGB-IR Face Verification and Person Re-identification
2024-11-28 15:38:15Kshitij Nikhal, Cedric Nimpa Fondje, Benjamin S. RigganAbstract
Cross-spectral biometrics, such as matching imagery of faces or persons from visible (RGB) and infrared (IR) bands, have rapidly advanced over the last decade due to increasing sensitivity, size, quality, and ubiquity of IR focal plane arrays and enhanced analytics beyond the visible spectrum. Current techniques for mitigating large spectral disparities between RGB and IR imagery often include learning a discriminative common subspace by exploiting precisely curated data acquired from multiple spectra. Although there are challenges with determining robust architectures for extracting common information, a critical limitation for supervised methods is poor scalability in terms of acquiring labeled data. Therefore, we propose a novel unsupervised cross-spectral framework that combines (1) a new pseudo triplet loss with cross-spectral voting, (2) a new cross-spectral attention network leveraging multiple subspaces, and (3) structured sparsity to perform more discriminative cross-spectral clustering. We extensively compare our proposed RGB-IR biometric learning framework (and its individual components) with recent and previous state-of-the-art models on two challenging benchmark datasets: DEVCOM Army Research Laboratory Visible-Thermal Face Dataset (ARL-VTF) and RegDB person re-identification dataset, and, in some cases, achieve performance superior to completely supervised methods.
Abstract (translated)
URL
https://arxiv.org/abs/2411.19215
PDF
-
Anti-Forgetting Adaptation for Unsupervised Person Re-identification
2024-11-22 03:05:06Hao Chen, Francois Bremond, Nicu Sebe, Shiliang ZhangAbstract
Regular unsupervised domain adaptive person re-identification (ReID) focuses on adapting a model from a source domain to a fixed target domain. However, an adapted ReID model can hardly retain previously-acquired knowledge and generalize to unseen data. In this paper, we propose a Dual-level Joint Adaptation and Anti-forgetting (DJAA) framework, which incrementally adapts a model to new domains without forgetting source domain and each adapted target domain. We explore the possibility of using prototype and instance-level consistency to mitigate the forgetting during the adaptation. Specifically, we store a small number of representative image samples and corresponding cluster prototypes in a memory buffer, which is updated at each adaptation step. With the buffered images and prototypes, we regularize the image-to-image similarity and image-to-prototype similarity to rehearse old knowledge. After the multi-step adaptation, the model is tested on all seen domains and several unseen domains to validate the generalization ability of our method. Extensive experiments demonstrate that our proposed method significantly improves the anti-forgetting, generalization and backward-compatible ability of an unsupervised person ReID model.
Abstract (translated)
常规的无监督领域适应性人员重新识别（ReID）主要集中在将模型从源域迁移到固定的靶域上。然而，经过适应后的ReID模型很难保持之前获得的知识，并且难以推广到未见过的数据中。在本文中，我们提出了一种双重级联适应与抗遗忘(DJAA)框架，该框架能够在不忘记源域和每个已适应的靶域知识的情况下，逐步地将模型迁移到新的领域上。我们探索了利用原型和实例层次的一致性来减轻适应过程中遗忘的可能性。具体来说，我们将少量代表性图像样本及其对应的聚类原型存储在一个内存缓冲区中，并在每次适应步骤时更新该缓冲区。借助于缓存的图像与原型，我们通过调节图像到图像相似性和图像到原型相似性来复习旧知识。经过多步适应后，模型将在所有见过的域和一些未见的域上进行测试以验证方法的泛化能力。大量的实验表明，我们提出的方法显著提高了无监督人员ReID模型的抗遗忘、泛化及向后兼容的能力。
URL
https://arxiv.org/abs/2411.14695
PDF
-
Generalizable Person Re-identification via Balancing Alignment and Uniformity
2024-11-18 11:13:30Yoonki Cho, Jaeyoon Kim, Woo Jae Kim, Junsik Jung, Sung-eui YoonAbstract
Domain generalizable person re-identification (DG re-ID) aims to learn discriminative representations that are robust to distributional shifts. While data augmentation is a straightforward solution to improve generalization, certain augmentations exhibit a polarized effect in this task, enhancing in-distribution performance while deteriorating out-of-distribution performance. In this paper, we investigate this phenomenon and reveal that it leads to sparse representation spaces with reduced uniformity. To address this issue, we propose a novel framework, Balancing Alignment and Uniformity (BAU), which effectively mitigates this effect by maintaining a balance between alignment and uniformity. Specifically, BAU incorporates alignment and uniformity losses applied to both original and augmented images and integrates a weighting strategy to assess the reliability of augmented samples, further improving the alignment loss. Additionally, we introduce a domain-specific uniformity loss that promotes uniformity within each source domain, thereby enhancing the learning of domain-invariant features. Extensive experimental results demonstrate that BAU effectively exploits the advantages of data augmentation, which previous studies could not fully utilize, and achieves state-of-the-art performance without requiring complex training procedures. The code is available at \url{this https URL}.
Abstract (translated)
领域通用的人体重新识别（DG re-ID）旨在学习能够抵御分布变化的判别表示。虽然数据增强是提高泛化能力的一种直接解决方案，但在某些情况下，这些增强方法表现出了一种两极分化的效果，在提升模型在训练集上的表现的同时却损害了其在未见数据集上的表现。在这篇论文中，我们研究了这一现象并揭示它导致了不均匀的稀疏表示空间。为了解决这个问题，我们提出了一种新的框架——平衡对齐和一致性的方法（BAU），通过维持对齐与一致性之间的平衡来有效地缓解这种效果。具体来说，BAU结合了应用于原始图像和增强图像上的对齐损失和一致性损失，并集成了一种权重策略以评估增强样本的可靠性，进一步提高了对齐损失的效果。此外，我们还引入了一种领域特定的一致性损失，它在每个源域内促进一致性，从而增强了学习领域的不变特征的能力。大量的实验结果表明，BAU有效地利用了数据增强的优势，这是以往研究未能充分挖掘出来的，并且实现了最先进的性能，而无需复杂的训练过程。代码可在\url{this https URL}获取。
URL
https://arxiv.org/abs/2411.11471
PDF
-
Skeleton-Guided Spatial-Temporal Feature Learning for Video-Based Visible-Infrared Person Re-Identification
2024-11-17 13:18:05Wenjia Jiang, Xiaoke Zhu, Jiakang Gao, Di LiaoAbstract
Video-based visible-infrared person re-identification (VVI-ReID) is challenging due to significant modality feature discrepancies. Spatial-temporal information in videos is crucial, but the accuracy of spatial-temporal information is often influenced by issues like low quality and occlusions in videos. Existing methods mainly focus on reducing modality differences, but pay limited attention to improving spatial-temporal features, particularly for infrared videos. To address this, we propose a novel Skeleton-guided spatial-Temporal feAture leaRning (STAR) method for VVI-ReID. By using skeleton information, which is robust to issues such as poor image quality and occlusions, STAR improves the accuracy of spatial-temporal features in videos of both modalities. Specifically, STAR employs two levels of skeleton-guided strategies: frame level and sequence level. At the frame level, the robust structured skeleton information is used to refine the visual features of individual frames. At the sequence level, we design a feature aggregation mechanism based on skeleton key points graph, which learns the contribution of different body parts to spatial-temporal features, further enhancing the accuracy of global features. Experiments on benchmark datasets demonstrate that STAR outperforms state-of-the-art methods. Code will be open source soon.
Abstract (translated)
基于视频的可见光-红外行人重识别（VVI-ReID）因其显著的模态特征差异而具有挑战性。视频中的时空信息至关重要，但其准确性常受视频质量低和遮挡等问题的影响。现有方法主要集中在减少模态差异上，但对于改善时空特性，尤其是对于红外视频方面的关注较少。为了解决这一问题，我们提出了一种用于VVI-ReID的新型骨骼引导的空间-时间特征学习（STAR）方法。通过使用对图像质量问题和遮挡具有鲁棒性的骨架信息，STAR提高了两种模态视频中空间-时间特性的准确性。具体而言，STAR采用了两个层次的骨骼引导策略：帧级和序列级。在帧级上，利用鲁棒的结构化骨架信息来优化单个帧的视觉特征；在序列级上，我们设计了一种基于骨架关键点图的特征聚合机制，该机制学习不同身体部位对空间-时间特性贡献的学习，进一步提高了全局特性的准确性。基准数据集上的实验表明，STAR优于现有最先进的方法。代码即将开源。
URL
https://arxiv.org/abs/2411.11069
PDF
-
DLCR: A Generative Data Expansion Framework via Diffusion for Clothes-Changing Person Re-ID
2024-11-11 18:28:33Nyle Siddiqui, Florinel Alin Croitoru, Gaurav Kumar Nayak, Radu Tudor Ionescu, Mubarak ShahAbstract
With the recent exhibited strength of generative diffusion models, an open research question is \textit{if images generated by these models can be used to learn better visual representations}. While this generative data expansion may suffice for easier visual tasks, we explore its efficacy on a more difficult discriminative task: clothes-changing person re-identification (CC-ReID). CC-ReID aims to match people appearing in non-overlapping cameras, even when they change their clothes across cameras. Not only are current CC-ReID models constrained by the limited diversity of clothing in current CC-ReID datasets, but generating additional data that retains important personal features for accurate identification is a current challenge. To address this issue we propose DLCR, a novel data expansion framework that leverages pre-trained diffusion and large language models (LLMs) to accurately generate diverse images of individuals in varied attire. We generate additional data for five benchmark CC-ReID datasets (PRCC, CCVID, LaST, VC-Clothes, and LTCC) and \textbf{increase their clothing diversity by \boldmath{$10$}x, totaling over \boldmath{$2.1$}M images generated}. DLCR employs diffusion-based text-guided inpainting, conditioned on clothing prompts constructed using LLMs, to generate synthetic data that only modifies a subject's clothes while preserving their personally identifiable features. With this massive increase in data, we introduce two novel strategies - progressive learning and test-time prediction refinement - that respectively reduce training time and further boosts CC-ReID performance. On the PRCC dataset, we obtain a large top-1 accuracy improvement of $11.3\%$ by training CAL, a previous state of the art (SOTA) method, with DLCR-generated data. We publicly release our code and generated data for each dataset here: \url{this https URL}.
Abstract (translated)
随着生成扩散模型近期展现出的强大能力，一个开放的研究问题是**这些模型生成的图像是否可以用于学习更好的视觉表示**。虽然这种生成数据扩展可能对于较简单的视觉任务已经足够，我们探讨了它在更具挑战性的辨别任务上的有效性：换装行人重识别（CC-ReID）。CC-ReID旨在匹配出现在不同摄像头中的人员，即使他们在不同摄像头中更换了衣服也能进行匹配。当前的CC-ReID模型不仅受到现有CC-ReID数据集中服装多样性有限的限制，而且生成能够保留重要个人特征以实现准确识别的额外数据也是一个当前的挑战。为了解决这个问题，我们提出了DLCR，一个全新的数据扩展框架，它利用预训练的扩散模型和大型语言模型（LLMs）来准确地生成穿着不同服饰的人们的多样化图像。我们在五个基准CC-ReID数据集（PRCC、CCVID、LaST、VC-Clothes 和 LTCC）上生成了额外的数据，并**将这些数据集中服装的多样性提高了10倍，总共生成超过210万张图片**。DLCR采用基于扩散的文字引导式图像修复技术，以LLMs构建的服饰提示为条件来生成合成数据，仅修改主题的衣服，同时保留其个人可识别特征。通过这一大量增加的数据，我们引入了两种新的策略——渐进学习和测试时间预测改进，分别减少了训练时间和进一步提升了CC-ReID性能。在PRCC数据集上，通过使用DLCR生成的数据训练前一最佳方法（SOTA）CAL，我们获得了11.3%的top-1准确率提升。我们的代码和为每个数据集生成的数据在此公开发布：\url{this https URL}。
URL
https://arxiv.org/abs/2411.07205
PDF
-
Dynamic Textual Prompt For Rehearsal-free Lifelong Person Re-identification
2024-11-09 00:57:19Hongyu Chen, Bingliang Jiao, Wenxuan Wang, Peng WangAbstract
Lifelong person re-identification attempts to recognize people across cameras and integrate new knowledge from continuous data streams. Key challenges involve addressing catastrophic forgetting caused by parameter updating and domain shift, and maintaining performance in seen and unseen domains. Many previous works rely on data memories to retain prior samples. However, the amount of retained data increases linearly with the number of training domains, leading to continually increasing memory consumption. Additionally, these methods may suffer significant performance degradation when data preservation is prohibited due to privacy concerns. To address these limitations, we propose using textual descriptions as guidance to encourage the ReID model to learn cross-domain invariant features without retaining samples. The key insight is that natural language can describe pedestrian instances with an invariant style, suggesting a shared textual space for any pedestrian images. By leveraging this shared textual space as an anchor, we can prompt the ReID model to embed images from various domains into a unified semantic space, thereby alleviating catastrophic forgetting caused by domain shifts. To achieve this, we introduce a task-driven dynamic textual prompt framework in this paper. This model features a dynamic prompt fusion module, which adaptively constructs and fuses two different textual prompts as anchors. This effectively guides the ReID model to embed images into a unified semantic space. Additionally, we design a text-visual feature alignment module to learn a more precise mapping between fine-grained visual and textual features. We also developed a learnable knowledge distillation module that allows our model to dynamically balance retaining existing knowledge with acquiring new knowledge. Extensive experiments demonstrate that our method outperforms SOTAs under various settings.
Abstract (translated)
终身行人重识别旨在跨摄像头识别人员，并整合连续数据流中的新知识。关键挑战包括解决由参数更新导致的灾难性遗忘和领域转移问题，以及在已见与未见过的领域中保持性能。许多先前的工作依赖于数据记忆来保留之前的样本。然而，随着训练领域的增加，所保留的数据量线性增长，导致内存消耗持续增加。此外，由于隐私方面的顾虑禁止保存数据时，这些方法可能会遭受显著的性能下降。为了解决这些问题，我们提出使用文本描述作为指导，促使ReID模型学习跨域不变特征而不必保留样本。核心见解是自然语言可以以一种不变的方式描述行人实例，表明存在一个任何行人都共享的文本空间。通过利用这个共享的文本空间作为锚点，我们可以提示ReID模型将来自不同领域的图像嵌入到统一的语义空间中，从而缓解由领域转移导致的灾难性遗忘问题。为了实现这一目标，本文引入了一种任务驱动的动态文本提示框架。该模型具有一种动态提示融合模块，能够自适应地构建并融合两种不同的文本提示作为锚点。这有效地引导ReID模型将图像嵌入到一个统一的语义空间中。此外，我们设计了一个文本视觉特征对齐模块，以学习更精确的细粒度视觉与文本特征之间的映射关系。我们还开发了一种可学习的知识蒸馏模块，使我们的模型能够动态平衡保留已有知识和获取新知识的能力。广泛的实验表明，在各种设置下，我们的方法优于现有的最先进技术（SOTAs）。
URL
https://arxiv.org/abs/2411.06023
PDF
-
Pose-Transformation and Radial Distance Clustering for Unsupervised Person Re-identification
2024-11-06 20:55:30Siddharth Seth, Akash Sonth, Anirban ChakrabortyAbstract
Person re-identification (re-ID) aims to tackle the problem of matching identities across non-overlapping cameras. Supervised approaches require identity information that may be difficult to obtain and are inherently biased towards the dataset they are trained on, making them unscalable across domains. To overcome these challenges, we propose an unsupervised approach to the person re-ID setup. Having zero knowledge of true labels, our proposed method enhances the discriminating ability of the learned features via a novel two-stage training strategy. The first stage involves training a deep network on an expertly designed pose-transformed dataset obtained by generating multiple perturbations for each original image in the pose space. Next, the network learns to map similar features closer in the feature space using the proposed discriminative clustering algorithm. We introduce a novel radial distance loss, that attends to the fundamental aspects of feature learning - compact clusters with low intra-cluster and high inter-cluster variation. Extensive experiments on several large-scale re-ID datasets demonstrate the superiority of our method compared to state-of-the-art approaches.
Abstract (translated)
人员再识别（re-ID）旨在解决跨非重叠摄像头匹配身份的问题。监督方法需要身份信息，这些信息可能难以获取，并且本质上倾向于它们所训练的数据集，使得它们在不同领域中不可扩展。为克服这些挑战，我们提出了一种无监督的人员re-ID设置方法。我们的提议方法对真实标签没有任何先验知识，并通过一种新颖的两阶段训练策略来增强学习特征的区分能力。第一阶段涉及在一个由生成每个原始图像的多个人体姿态变换而得的数据集上训练深度网络。接下来，该网络利用所提出的判别聚类算法，在特征空间中将相似特征映射得更近。我们引入了一种新颖的径向距离损失函数，它关注于特征学习的基本方面——即具有低簇内和高簇间变异性的紧凑簇。在多个大规模re-ID数据集上的广泛实验表明，我们的方法优于最先进的方法。
URL
https://arxiv.org/abs/2411.04255
PDF
-
Multiple Information Prompt Learning for Cloth-Changing Person Re-Identification
2024-11-01 03:08:10Shengxun Wei, Zan Gao, Yibo Zhao, Weili GuanAbstract
Cloth-changing person re-identification is a subject closer to the real world, which focuses on solving the problem of person re-identification after pedestrians change clothes. The primary challenge in this field is to overcome the complex interplay between intra-class and inter-class variations and to identify features that remain unaffected by changes in appearance. Sufficient data collection for model training would significantly aid in addressing this problem. However, it is challenging to gather diverse datasets in practice. Current methods focus on implicitly learning identity information from the original image or introducing additional auxiliary models, which are largely limited by the quality of the image and the performance of the additional model. To address these issues, inspired by prompt learning, we propose a novel multiple information prompt learning (MIPL) scheme for cloth-changing person ReID, which learns identity robust features through the common prompt guidance of multiple messages. Specifically, the clothing information stripping (CIS) module is designed to decouple the clothing information from the original RGB image features to counteract the influence of clothing appearance. The Bio-guided attention (BGA) module is proposed to increase the learning intensity of the model for key information. A dual-length hybrid patch (DHP) module is employed to make the features have diverse coverage to minimize the impact of feature bias. Extensive experiments demonstrate that the proposed method outperforms all state-of-the-art methods on the LTCC, Celeb-reID, Celeb-reID-light, and CSCC datasets, achieving rank-1 scores of 74.8%, 73.3%, 66.0%, and 88.1%, respectively. When compared to AIM (CVPR23), ACID (TIP23), and SCNet (MM23), MIPL achieves rank-1 improvements of 11.3%, 13.8%, and 7.9%, respectively, on the PRCC dataset.
Abstract (translated)
换装行人重识别是一个更贴近现实世界的主题，专注于解决行人在更换服装后的人重新识别问题。这一领域的首要挑战在于克服类内和类间变化的复杂相互作用，并识别出不受外观改变影响的特征。为模型训练收集足够的数据将显著有助于解决这个问题。然而，在实践中收集多样化的数据集具有挑战性。当前的方法主要集中在从原始图像中隐式学习身份信息或引入额外的辅助模型，这些方法大多受到图像质量和附加模型性能的限制。为了应对这些问题，受提示学习启发，我们提出了一种针对换装行人重识别的新颖多信息提示学习（MIPL）方案，通过多个消息的共同提示指导来学习鲁棒的身份特征。具体而言，设计了服装信息剥离（CIS）模块，以从原始RGB图像特征中解耦服装信息，从而抵消服装外观的影响。提出了生物引导注意力（BGA）模块，增加模型对关键信息的学习强度。采用双长度混合补丁（DHP）模块使特征具有多样化的覆盖范围，以最小化特征偏差的影响。广泛的实验表明，所提出的方法在LTCC、Celeb-reID、Celeb-reID-light和CSCC数据集上优于所有最先进的方法，分别实现了74.8%、73.3%、66.0%和88.1%的rank-1分数。与AIM（CVPR23）、ACID（TIP23）和SCNet（MM23）相比，在PRCC数据集上，MIPL在rank-1得分上分别提高了11.3%、13.8%和7.9%。
URL
https://arxiv.org/abs/2411.00330
PDF
-
ReMix: Training Generalized Person Re-identification on a Mixture of Data
2024-10-29 10:57:03Timur Mamedov, Anton Konushin, Vadim KonushinAbstract
Modern person re-identification (Re-ID) methods have a weak generalization ability and experience a major accuracy drop when capturing environments change. This is because existing multi-camera Re-ID datasets are limited in size and diversity, since such data is difficult to obtain. At the same time, enormous volumes of unlabeled single-camera records are available. Such data can be easily collected, and therefore, it is more diverse. Currently, single-camera data is used only for self-supervised pre-training of Re-ID methods. However, the diversity of single-camera data is suppressed by fine-tuning on limited multi-camera data after pre-training. In this paper, we propose ReMix, a generalized Re-ID method jointly trained on a mixture of limited labeled multi-camera and large unlabeled single-camera data. Effective training of our method is achieved through a novel data sampling strategy and new loss functions that are adapted for joint use with both types of data. Experiments show that ReMix has a high generalization ability and outperforms state-of-the-art methods in generalizable person Re-ID. To the best of our knowledge, this is the first work that explores joint training on a mixture of multi-camera and single-camera data in person Re-ID.
Abstract (translated)
现代人员重识别（Re-ID）方法具有较弱的泛化能力，当捕捉环境发生变化时会遇到显著的准确率下降。这是因为现有的多摄像头Re-ID数据集在规模和多样性上有限，因为这类数据难以获取。同时，有大量的未标注单摄像头记录可供使用。这种数据易于收集，因此更加多样化。目前，单摄像头数据仅用于Re-ID方法的自监督预训练。然而，在预训练后用有限的多摄像头数据进行微调会抑制单摄像头数据的多样性。在本文中，我们提出了ReMix，这是一种在有限标记的多摄像头和大量的未标注单摄像头数据混合上共同训练的泛化Re-ID方法。通过一种新颖的数据采样策略和适应于两种类型数据联合使用的新的损失函数，实现了对我们的方法的有效训练。实验表明，ReMix具有较高的泛化能力，在可泛化的人员重识别方面优于最先进的方法。据我们所知，这是首次探索在人员重识别中使用多摄像头与单摄像头数据混合进行共同训练的工作。
URL
https://arxiv.org/abs/2410.21938
PDF
-
Discriminative Pedestrian Features and Gated Channel Attention for Clothes-Changing Person Re-Identification
2024-10-29 02:12:46Yongkang Ding, Rui Mao, Hanyue Zhu, Anqi Wang, Liyan ZhangAbstract
In public safety and social life, the task of Clothes-Changing Person Re-Identification (CC-ReID) has become increasingly significant. However, this task faces considerable challenges due to appearance changes caused by clothing alterations. Addressing this issue, this paper proposes an innovative method for disentangled feature extraction, effectively extracting discriminative features from pedestrian images that are invariant to clothing. This method leverages pedestrian parsing techniques to identify and retain features closely associated with individual identity while disregarding the variable nature of clothing attributes. Furthermore, this study introduces a gated channel attention mechanism, which, by adjusting the network's focus, aids the model in more effectively learning and emphasizing features critical for pedestrian identity recognition. Extensive experiments conducted on two standard CC-ReID datasets validate the effectiveness of the proposed approach, with performance surpassing current leading solutions. The Top-1 accuracy under clothing change scenarios on the PRCC and VC-Clothes datasets reached 64.8% and 83.7%, respectively.
Abstract (translated)
在公共安全和社会生活中，衣服变换行人重识别（CC-ReID）任务变得越来越重要。然而，由于服装变化导致的外观变化，这一任务面临着相当大的挑战。为了解决这个问题，本文提出了一种创新的分离特征提取方法，能够有效地从行人的图像中提取出不受服装影响的判别性特征。该方法利用行人解析技术来识别和保留与个体身份密切相关的特征，同时忽略服装属性的变化。此外，本研究引入了门控通道注意力机制，通过调整网络的关注点，帮助模型更有效地学习并强调对于行人身份识别至关重要的特征。在两个标准CC-ReID数据集上进行的广泛实验验证了所提出方法的有效性，其性能超越了当前领先的解决方案。在PRCC和VC-Clothes数据集上的衣服变化场景下，Top-1准确率分别达到了64.8%和83.7%。
URL
https://arxiv.org/abs/2410.21663
PDF
-
Exploring Stronger Transformer Representation Learning for Occluded Person Re-Identificatio
2024-10-21 03:17:25Zhangjian Ji, Donglin Cheng, Kai FengAbstract
Due to some complex factors (e.g., occlusion, pose variation and diverse camera perspectives), extracting stronger feature representation in person re-identification remains a challenging task. In this paper, we proposed a novel self-supervision and supervision combining transformer-based person re-identification framework, namely SSSC-TransReID. Different from the general transformer-based person re-identification models, we designed a self-supervised contrastive learning branch, which can enhance the feature representation for person re-identification without negative samples or additional pre-training. In order to train the contrastive learning branch, we also proposed a novel random rectangle mask strategy to simulate the occlusion in real scenes, so as to enhance the feature representation for occlusion. Finally, we utilized the joint-training loss function to integrate the advantages of supervised learning with ID tags and self-supervised contrastive learning without negative samples, which can reinforce the ability of our model to excavate stronger discriminative features, especially for occlusion. Extensive experimental results on several benchmark datasets show our proposed model obtains superior Re-ID performance consistently and outperforms the state-of-the-art ReID methods by large margins on the mean average accuracy (mAP) and Rank-1 accuracy.
Abstract (translated)
URL
https://arxiv.org/abs/2410.15613
PDF
-
GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learning
2024-10-20 03:45:50Haiwen Diao, Ying Zhang, Shang Gao, Jiawen Zhu, Long Chen, Huchuan LuAbstract
Cross-modal metric learning is a prominent research topic that bridges the semantic heterogeneity between vision and language. Existing methods frequently utilize simple cosine or complex distance metrics to transform the pairwise features into a similarity score, which suffers from an inadequate or inefficient capability for distance measurements. Consequently, we propose a Generalized Structural Sparse Function to dynamically capture thorough and powerful relationships across modalities for pair-wise similarity learning while remaining concise but efficient. Specifically, the distance metric delicately encapsulates two formats of diagonal and block-diagonal terms, automatically distinguishing and highlighting the cross-channel relevancy and dependency inside a structured and organized topology. Hence, it thereby empowers itself to adapt to the optimal matching patterns between the paired features and reaches a sweet spot between model complexity and capability. Extensive experiments on cross-modal and two extra uni-modal retrieval tasks (image-text retrieval, person re-identification, fine-grained image retrieval) have validated its superiority and flexibility over various popular retrieval frameworks. More importantly, we further discover that it can be seamlessly incorporated into multiple application scenarios, and demonstrates promising prospects from Attention Mechanism to Knowledge Distillation in a plug-and-play manner. Our code is publicly available at: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2410.15266
PDF
-
Pseudo-label Refinement for Improving Self-Supervised Learning Systems
2024-10-18 07:47:59Zia-ur-Rehman, Arif Mahmood, Wenxiong KangAbstract
Self-supervised learning systems have gained significant attention in recent years by leveraging clustering-based pseudo-labels to provide supervision without the need for human annotations. However, the noise in these pseudo-labels caused by the clustering methods poses a challenge to the learning process leading to degraded performance. In this work, we propose a pseudo-label refinement (SLR) algorithm to address this issue. The cluster labels from the previous epoch are projected to the current epoch cluster-labels space and a linear combination of the new label and the projected label is computed as a soft refined label containing the information from the previous epoch clusters as well as from the current epoch. In contrast to the common practice of using the maximum value as a cluster/class indicator, we employ hierarchical clustering on these soft pseudo-labels to generate refined hard-labels. This approach better utilizes the information embedded in the soft labels, outperforming the simple maximum value approach for hard label generation. The effectiveness of the proposed SLR algorithm is evaluated in the context of person re-identification (Re-ID) using unsupervised domain adaptation (UDA). Experimental results demonstrate that the modified Re-ID baseline, incorporating the SLR algorithm, achieves significantly improved mean Average Precision (mAP) performance in various UDA tasks, including real-to-synthetic, synthetic-to-real, and different real-to-real scenarios. These findings highlight the efficacy of the SLR algorithm in enhancing the performance of self-supervised learning systems.
Abstract (translated)
URL
https://arxiv.org/abs/2410.14242
PDF
-
CCUP: A Controllable Synthetic Data Generation Pipeline for Pretraining Cloth-Changing Person Re-Identification Models
2024-10-17 14:04:02Yujian Zhao, Chengru Wu, Yinong Xu, Xuanzheng Du, Ruiyu Li, Guanglin NiuAbstract
Cloth-changing person re-identification (CC-ReID), also known as Long-Term Person Re-Identification (LT-ReID) is a critical and challenging research topic in computer vision that has recently garnered significant attention. However, due to the high cost of constructing CC-ReID data, the existing data-driven models are hard to train efficiently on limited data, causing overfitting issue. To address this challenge, we propose a low-cost and efficient pipeline for generating controllable and high-quality synthetic data simulating the surveillance of real scenarios specific to the CC-ReID task. Particularly, we construct a new self-annotated CC-ReID dataset named Cloth-Changing Unreal Person (CCUP), containing 6,000 IDs, 1,179,976 images, 100 cameras, and 26.5 outfits per individual. Based on this large-scale dataset, we introduce an effective and scalable pretrain-finetune framework for enhancing the generalization capabilities of the traditional CC-ReID models. The extensive experiments demonstrate that two typical models namely TransReID and FIRe^2, when integrated into our framework, outperform other state-of-the-art models after pretraining on CCUP and finetuning on the benchmarks such as PRCC, VC-Clothes and NKUP. The CCUP is available at: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2410.13567
PDF
