Abstract
With the recent exhibited strength of generative diffusion models, an open research question is \textit{if images generated by these models can be used to learn better visual representations}. While this generative data expansion may suffice for easier visual tasks, we explore its efficacy on a more difficult discriminative task: clothes-changing person re-identification (CC-ReID). CC-ReID aims to match people appearing in non-overlapping cameras, even when they change their clothes across cameras. Not only are current CC-ReID models constrained by the limited diversity of clothing in current CC-ReID datasets, but generating additional data that retains important personal features for accurate identification is a current challenge. To address this issue we propose DLCR, a novel data expansion framework that leverages pre-trained diffusion and large language models (LLMs) to accurately generate diverse images of individuals in varied attire. We generate additional data for five benchmark CC-ReID datasets (PRCC, CCVID, LaST, VC-Clothes, and LTCC) and \textbf{increase their clothing diversity by \boldmath{$10$}x, totaling over \boldmath{$2.1$}M images generated}. DLCR employs diffusion-based text-guided inpainting, conditioned on clothing prompts constructed using LLMs, to generate synthetic data that only modifies a subject's clothes while preserving their personally identifiable features. With this massive increase in data, we introduce two novel strategies - progressive learning and test-time prediction refinement - that respectively reduce training time and further boosts CC-ReID performance. On the PRCC dataset, we obtain a large top-1 accuracy improvement of $11.3\%$ by training CAL, a previous state of the art (SOTA) method, with DLCR-generated data. We publicly release our code and generated data for each dataset here: \url{this https URL}.
Abstract (translated)
随着生成扩散模型近期展现出的强大能力，一个开放的研究问题是**这些模型生成的图像是否可以用于学习更好的视觉表示**。虽然这种生成数据扩展可能对于较简单的视觉任务已经足够，我们探讨了它在更具挑战性的辨别任务上的有效性：换装行人重识别（CC-ReID）。CC-ReID旨在匹配出现在不同摄像头中的人员，即使他们在不同摄像头中更换了衣服也能进行匹配。当前的CC-ReID模型不仅受到现有CC-ReID数据集中服装多样性有限的限制，而且生成能够保留重要个人特征以实现准确识别的额外数据也是一个当前的挑战。为了解决这个问题，我们提出了DLCR，一个全新的数据扩展框架，它利用预训练的扩散模型和大型语言模型（LLMs）来准确地生成穿着不同服饰的人们的多样化图像。我们在五个基准CC-ReID数据集（PRCC、CCVID、LaST、VC-Clothes 和 LTCC）上生成了额外的数据，并**将这些数据集中服装的多样性提高了10倍，总共生成超过210万张图片**。DLCR采用基于扩散的文字引导式图像修复技术，以LLMs构建的服饰提示为条件来生成合成数据，仅修改主题的衣服，同时保留其个人可识别特征。通过这一大量增加的数据，我们引入了两种新的策略——渐进学习和测试时间预测改进，分别减少了训练时间和进一步提升了CC-ReID性能。在PRCC数据集上，通过使用DLCR生成的数据训练前一最佳方法（SOTA）CAL，我们获得了11.3%的top-1准确率提升。我们的代码和为每个数据集生成的数据在此公开发布：\url{this https URL}。
URL
https://arxiv.org/abs/2411.07205
