Abstract
Recent generative data augmentation methods conditioned on both image and text prompts struggle to balance between fidelity and diversity, as it is challenging to preserve essential image details while aligning with varied text prompts. This challenge arises because representations in the synthesis process often become entangled with non-essential input image attributes such as environmental contexts, creating conflicts with text prompts intended to modify these elements. To address this, we propose a personalized image generation framework that uses a salient concept-aware image embedding model to reduce the influence of irrelevant visual details during the synthesis process, thereby maintaining intuitive alignment between image and text inputs. By generating images that better preserve class-discriminative features with additional controlled variations, our framework effectively enhances the diversity of training datasets and thereby improves the robustness of downstream models. Our approach demonstrates superior performance across eight fine-grained vision datasets, outperforming state-of-the-art augmentation methods with averaged classification accuracy improvements by 0.73% and 6.5% under conventional and long-tail settings, respectively.
Abstract (translated)
最近的基于图像和文本提示的生成数据增强方法在保真度与多样性之间难以取得平衡，因为这些方法往往很难在保留图像关键细节的同时适应多样化的文本提示。这一挑战源于合成过程中表示的纠缠问题，即重要图像属性（如核心视觉特征）经常与其他非本质输入属性（例如环境背景信息）混杂在一起，导致与旨在修改这些元素的文本提示产生冲突。 为解决上述难题，我们提出了一种个性化图像生成框架，该框架采用显著概念感知的图像嵌入模型，减少合成过程中无关视觉细节的影响。这样一来，可以更好地保持图像和文本输入之间直观一致的关系。通过在保留类别判别性特征的同时引入额外控制变化，我们的方法能够有效增加训练数据集的多样性，并提升下游模型的鲁棒性。 实验结果表明，在八个细粒度视觉数据集中，相较于现有先进增强技术，我们的方法表现出色：常规设置下分类准确率平均提高了0.73%，长尾分布场景下则提升了6.5%。
URL
https://arxiv.org/abs/2510.15194
