Abstract
Image Generation models are a trending topic nowadays, with many people utilizing Artificial Intelligence models in order to generate images. There are many such models which, given a prompt of a text, will generate an image which depicts said prompt. There are many image generation models, such as Latent Diffusion Models, Denoising Diffusion Probabilistic Models, Generative Adversarial Networks and many more. When generating images, these models can generate sensitive image data, which can be threatening to privacy or may violate copyright laws of private entities. Machine unlearning aims at removing the influence of specific data subsets from the trained models and in the case of image generation models, remove the influence of a concept such that the model is unable to generate said images of the concept when prompted. Conventional retraining of the model can take upto days, hence fast algorithms are the need of the hour. In this paper we propose an algorithm that aims to remove the influence of concepts in diffusion models through updating the gradients of the final layers of the text encoders. Using a weighted loss function, we utilize backpropagation in order to update the weights of the final layers of the Text Encoder componet of the Stable Diffusion Model, removing influence of the concept from the text-image embedding space, such that when prompted, the result is an image not containing the concept. The weighted loss function makes use of Textual Inversion and Low-Rank this http URL perform our experiments on Latent Diffusion Models, namely the Stable Diffusion v2 model, with an average concept unlearning runtime of 50 seconds using 4-5 images.
Abstract (translated)
图像生成模型是当今热门的话题，许多人利用人工智能模型来生成图片。这些模型中的许多在收到文本提示后能够生成相应的图片。目前有许多图像生成模型，如潜在扩散模型、去噪扩散概率模型以及生成对抗网络等。当使用这些模型进行图像生成时，可能会产生涉及隐私或可能侵犯私有实体版权的敏感内容。 机器遗忘技术旨在从训练好的模型中移除特定数据集的影响，在图像生成模型的情况下，则是为了去除某些概念的影响，使得在收到相应提示后，模型无法生成带有该概念的图片。传统的重新训练过程可能需要数天时间来完成，因此快速算法变得尤为重要。在这篇论文中，我们提出了一种旨在通过更新文本编码器最终层的梯度来移除扩散模型中的特定概念影响的算法。使用加权损失函数并通过反向传播方法来更新稳定扩散模型（Stable Diffusion）文本编码组件最终层的权重，在此过程中从文本-图像嵌入空间中去除该概念的影响，使得当再次收到相同提示时生成的结果图片不再包含该概念。 我们在潜在扩散模型上进行了实验验证，具体使用的是Stable Diffusion v2版本的模型。我们的实验表明，通过使用4到5张相关图片进行训练，在平均情况下可以实现大约50秒的概念遗忘运行时间。
URL
https://arxiv.org/abs/2505.12395
