Paper Reading AI Learner

Diffusion Models with Adaptive Negative Sampling Without External Resources

2025-08-05 00:45:54
Alakh Desai, Nuno Vasconcelos

Abstract

Diffusion models (DMs) have demonstrated an unparalleled ability to create diverse and high-fidelity images from text prompts. However, they are also well-known to vary substantially regarding both prompt adherence and quality. Negative prompting was introduced to improve prompt compliance by specifying what an image must not contain. Previous works have shown the existence of an ideal negative prompt that can maximize the odds of the positive prompt. In this work, we explore relations between negative prompting and classifier-free guidance (CFG) to develop a sampling procedure, {\it Adaptive Negative Sampling Without External Resources} (ANSWER), that accounts for both positive and negative conditions from a single prompt. This leverages the internal understanding of negation by the diffusion model to increase the odds of generating images faithful to the prompt. ANSWER is a training-free technique, applicable to any model that supports CFG, and allows for negative grounding of image concepts without an explicit negative prompts, which are lossy and incomplete. Experiments show that adding ANSWER to existing DMs outperforms the baselines on multiple benchmarks and is preferred by humans 2x more over the other methods.

Abstract (translated)

扩散模型(DM)已经展示了从文本提示生成多样且高质量图像的无与伦比的能力。然而,它们在遵循提示和质量方面也表现出相当大的差异性。为了解决这一问题,引入了负向提示来明确说明图像中不应包含的内容,从而提高对正向提示的遵从度。先前的研究表明存在一个理想的负向提示可以最大化正向提示的成功率。 在这项工作中,我们探讨了负向提示与无分类器指导(CFG)之间的关系,并开发了一种采样程序——“无需外部资源的自适应负样本生成”(ANSWER),该程序能够在单个提示下同时考虑正面和负面条件。此方法利用扩散模型对否定的理解能力来提高图像忠实于提示的概率。ANSWER是一种不需要训练的技术,适用于任何支持CFG的模型,并且可以在没有显式负向提示的情况下实现图像概念的负向锚定,而显式的负向提示往往是低效且不完整的。 实验结果表明,在现有DM中加入ANSWER在多个基准测试上超越了基线方法,并且人类更偏爱该方法2倍于其他方法。

URL

https://arxiv.org/abs/2508.02973

PDF

https://arxiv.org/pdf/2508.02973.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot