Abstract
The diffusion models (DMs) have demonstrated the remarkable capability of generating images via learning the noised score function of data distribution. Current DM sampling techniques typically rely on first-order Langevin dynamics at each noise level, with efforts concentrated on refining inter-level denoising strategies. While leveraging additional second-order Hessian geometry to enhance the sampling quality of Langevin is a common practice in Markov chain Monte Carlo (MCMC), the naive attempts to utilize Hessian geometry in high-dimensional DMs lead to quadratic-complexity computational costs, rendering them non-scalable. In this work, we introduce a novel Levenberg-Marquardt-Langevin (LML) method that approximates the diffusion Hessian geometry in a training-free manner, drawing inspiration from the celebrated Levenberg-Marquardt optimization algorithm. Our approach introduces two key innovations: (1) A low-rank approximation of the diffusion Hessian, leveraging the DMs' inherent structure and circumventing explicit quadratic-complexity computations; (2) A damping mechanism to stabilize the approximated Hessian. This LML approximated Hessian geometry enables the diffusion sampling to execute more accurate steps and improve the image generation quality. We further conduct a theoretical analysis to substantiate the approximation error bound of low-rank approximation and the convergence property of the damping mechanism. Extensive experiments across multiple pretrained DMs validate that the LML method significantly improves image generation quality, with negligible computational overhead.
Abstract (translated)
扩散模型（DM）通过学习数据分布的噪声得分函数，展示了生成图像的强大能力。目前的DM采样技术通常依赖于每个噪声水平上的第一阶Langevin动力学，并且努力集中在改进跨层级去噪策略上。尽管在马尔可夫链蒙特卡洛（MCMC）方法中利用额外的二阶Hessian几何来增强Langevin采样的质量是一种常见做法，但在高维DM中直接使用Hessian几何会导致二次复杂度的计算成本，使其无法扩展。 在这项工作中，我们引入了一种新的莱文伯格-马夸尔特-兰热姆（Levenberg-Marquardt-Langevin, LML）方法，该方法以无训练的方式近似扩散Hessian几何，借鉴了著名的Levenberg-Marquardt优化算法。我们的方法提出了两项关键创新： 1. 一种对扩散Hessian的低秩近似，利用DM固有的结构，并避免直接进行二次复杂度计算。 2. 一个阻尼机制来稳定近似的Hessian。 这种LML近似Hessian几何使扩散采样能够执行更准确的步骤并提高图像生成质量。我们进一步进行了理论分析以验证低秩近似误差界和阻尼机制的收敛性。在多个预训练DM上的广泛实验验证了LML方法显著提高了图像生成的质量，且计算开销几乎可以忽略不计。
URL
https://arxiv.org/abs/2505.24222
