Abstract
Large-scale text-to-image (T2I) diffusion models have revolutionized image generation, enabling the synthesis of highly detailed visuals from textual descriptions. However, these models may inadvertently generate inappropriate content, such as copyrighted works or offensive images. While existing methods attempt to eliminate specific unwanted concepts, they often fail to ensure complete removal, allowing the concept to reappear in subtle forms. For instance, a model may successfully avoid generating images in Van Gogh's style when explicitly prompted with 'Van Gogh', yet still reproduce his signature artwork when given the prompt 'Starry Night'. In this paper, we propose SAFER, a novel and efficient approach for thoroughly removing target concepts from diffusion models. At a high level, SAFER is inspired by the observed low-dimensional structure of the text embedding space. The method first identifies a concept-specific subspace $S_c$ associated with the target concept c. It then projects the prompt embeddings onto the complementary subspace of $S_c$, effectively erasing the concept from the generated images. Since concepts can be abstract and difficult to fully capture using natural language alone, we employ textual inversion to learn an optimized embedding of the target concept from a reference image. This enables more precise subspace estimation and enhances removal performance. Furthermore, we introduce a subspace expansion strategy to ensure comprehensive and robust concept erasure. Extensive experiments demonstrate that SAFER consistently and effectively erases unwanted concepts from diffusion models while preserving generation quality.
Abstract (translated)
大规模的文本到图像（T2I）扩散模型已经革新了图像生成技术，能够从文本描述中合成出高度详细的视觉内容。然而，这些模型可能会无意间生成不适当的内容，例如受版权保护的作品或冒犯性的图像。现有的方法试图消除特定的不需要的概念，但往往无法确保完全移除这些概念，导致它们以微妙的形式重新出现。例如，当明确提示为“Van Gogh”时，模型可能成功避免生成梵高风格的图像，但在收到提示“星夜”的情况下，仍然会复制他的标志性作品。在这篇论文中，我们提出了SAFER（Safe and Efficient Removal），这是一种新的且高效的从扩散模型中彻底移除目标概念的方法。 总体而言，SAFER受到了文本嵌入空间低维结构观察到的现象的启发。该方法首先识别与目标概念c相关的特定子空间$S_c$。然后将提示嵌入投影到$S_c$的补子空间上，从而有效地从生成的图像中擦除概念。由于概念可能很抽象且难以仅通过自然语言完全捕捉，我们使用文本反转技术来从参考图像中学习目标概念的优化嵌入，这使更精确的子空间估计成为可能，并增强了移除性能。 此外，我们引入了一种子空间扩展策略，以确保全面和稳健的概念擦除。广泛的实验表明，SAFER能够持续且有效地从扩散模型中移除不需要的概念，同时保持生成的质量。
URL
https://arxiv.org/abs/2503.16835
