Abstract
Person re-identification (re-id) models are vital in security surveillance systems, requiring transferable adversarial attacks to explore the vulnerabilities of them. Recently, vision-language models (VLM) based attacks have shown superior transferability by attacking generalized image and textual features of VLM, but they lack comprehensive feature disruption due to the overemphasis on discriminative semantics in integral representation. In this paper, we introduce the Attribute-aware Prompt Attack (AP-Attack), a novel method that leverages VLM's image-text alignment capability to explicitly disrupt fine-grained semantic features of pedestrian images by destroying attribute-specific textual embeddings. To obtain personalized textual descriptions for individual attributes, textual inversion networks are designed to map pedestrian images to pseudo tokens that represent semantic embeddings, trained in the contrastive learning manner with images and a predefined prompt template that explicitly describes the pedestrian attributes. Inverted benign and adversarial fine-grained textual semantics facilitate attacker in effectively conducting thorough disruptions, enhancing the transferability of adversarial examples. Extensive experiments show that AP-Attack achieves state-of-the-art transferability, significantly outperforming previous methods by 22.9% on mean Drop Rate in cross-model&dataset attack scenarios.
Abstract (translated)
人员重识别（Re-ID）模型在安全监控系统中至关重要，需要具有可转移性的对抗攻击来探索其漏洞。最近，基于视觉-语言模型（VLM）的攻击通过针对VLM中的通用图像和文本特征进行攻击展示了出色的可迁移性，但它们由于过分强调整体表示中的辨别语义而缺乏全面的功能破坏。 在本文中，我们介绍了属性感知提示攻击（AP-Attack），这是一种新颖的方法，它利用了VLM的图-文对齐能力来显式地破坏行人图像中的细粒度语义特征，通过摧毁特定于属性的文本嵌入。为了为每个单独的属性获得个性化的文本描述，设计了文本逆向网络将行人图像映射到表示语义嵌入的伪令牌上，并采用对比学习方式与图像和预先定义的提示模板一起训练，该模板明确描述了行人的属性特征。 通过这种逆向处理，生成的良性细粒度文本语义和对抗性细粒度文本语义帮助攻击者更有效地进行全面破坏，从而增强对抗样本的可迁移性。广泛的实验表明，AP-Attack在跨模型及数据集攻击场景中实现了最先进的可迁移性，比先前的方法平均Drop Rate指标高出了22.9%。 简而言之，通过引入属性感知提示攻击（AP-Attack），我们为提高行人重识别系统的对抗鲁棒性和安全性提供了一种创新而有效的方法。
URL
https://arxiv.org/abs/2502.19697
