Abstract
This paper proposes a dataset augmentation method by fine-tuning pre-trained diffusion models. Generating images using a pre-trained diffusion model with textual conditioning often results in domain discrepancy between real data and generated images. We propose a fine-tuning approach where we adapt the diffusion model by conditioning it with real images and novel text embeddings. We introduce a unique procedure called Mixing Visual Concepts (MVC) where we create novel text embeddings from image captions. The MVC enables us to generate multiple images which are diverse and yet similar to the real data enabling us to perform effective dataset augmentation. We perform comprehensive qualitative and quantitative evaluations with the proposed dataset augmentation approach showcasing both coarse-grained and finegrained changes in generated images. Our approach outperforms state-of-the-art augmentation techniques on benchmark classification tasks.
Abstract (translated)
本文提出了一种通过微调预训练扩散模型来实现数据集扩增的方法。使用具有文本条件的预训练扩散模型生成图像通常会导致真实数据与生成图像之间的领域差异。我们提出了一个微调方法，即通过用真实图像和新颖的文本嵌入对扩散模型进行条件设置来调整该模型。我们引入了一种名为“混合视觉概念”（MVC）的独特程序，在此过程中，我们从图像标题中创建新的文本嵌入。借助MVC，我们可以生成多种多样但又与真实数据相似的图像，从而实现有效的数据集扩增。我们对提出的这种数据集扩增方法进行了全面的定性和定量评估，展示了生成图像在粗粒度和细粒度上的变化。我们的方法在基准分类任务上超越了最先进的扩增技术。
URL
https://arxiv.org/abs/2412.15358
