Abstract
Pre-trained diffusion models have enabled significant advancements in All-in-One Restoration (AiOR), offering improved perceptual quality and generalization. However, diffusion-based restoration methods primarily rely on fine-tuning or Control-Net style modules to leverage the pre-trained diffusion model's priors for AiOR. In this work, we show that these pre-trained diffusion models inherently possess restoration behavior, which can be unlocked by directly learning prompt embeddings at the output of the text encoder. Interestingly, this behavior is largely inaccessible through text prompts and text-token embedding optimization. Furthermore, we observe that naive prompt learning is unstable because the forward noising process using degraded images is misaligned with the reverse sampling trajectory. To resolve this, we train prompts within a diffusion bridge formulation that aligns training and inference dynamics, enforcing a coherent denoising path from noisy degraded states to clean images. Building on these insights, we introduce our lightweight learned prompts on the pre-trained WAN video model and FLUX image models, converting them into high-performing restoration models. Extensive experiments demonstrate that our approach achieves competitive performance and generalization across diverse degradations, while avoiding fine-tuning and restoration-specific control modules.
Abstract (translated)
预训练扩散模型推动了全合一图像恢复（AiOR）领域的显著进步，提供了更优的感知质量和泛化能力。然而，基于扩散的恢复方法主要依赖微调或Control-Net风格模块来利用预训练扩散模型的先验知识。本研究表明，这些预训练扩散模型本身固有恢复行为，可通过直接在文本编码器输出端学习提示词嵌入来激活。有趣的是，这种行为难以通过文本提示或文本词元嵌入优化有效获取。此外，我们发现朴素提示词学习不稳定，因为使用退化图像的前向加噪过程与反向采样轨迹不匹配。为解决此问题，我们在扩散桥框架内训练提示词，使训练与推理动态对齐，强制从噪声退化状态到清晰图像形成连贯的降噪路径。基于这些见解，我们提出在预训练的WAN视频模型和FLUX图像模型上使用轻量级学习提示词，将其转化为高性能恢复模型。大量实验表明，该方法在多种退化场景下均具有竞争力的性能与泛化能力，同时避免了微调及特定恢复控制模块的需求。
URL
https://arxiv.org/abs/2604.04924
