Abstract
Diffusion-based generative image compression has demonstrated remarkable potential for achieving realistic reconstruction at ultra-low bitrates. The key to unlocking this potential lies in making the entire compression process content-adaptive, ensuring that the encoder's representation and the decoder's generative prior are dynamically aligned with the semantic and structural characteristics of the input image. However, existing methods suffer from three critical limitations that prevent effective content adaptation. First, isotropic quantization applies a uniform quantization step, failing to adapt to the spatially varying complexity of image content and creating a misalignment with the diffusion model's noise-dependent prior. Second, the information concentration bottleneck -- arising from the dimensional mismatch between the high-dimensional noisy latent and the diffusion decoder's fixed input -- prevents the model from adaptively preserving essential semantic information in the primary channels. Third, existing textual conditioning strategies either need significant textual bitrate overhead or rely on generic, content-agnostic textual prompts, thereby failing to provide adaptive semantic guidance efficiently. To overcome these limitations, we propose a content-adaptive diffusion-based image codec with three technical innovations: 1) an Uncertainty-Guided Adaptive Quantization method that learns spatial uncertainty maps to adaptively align quantization distortion with content characteristics; 2) an Auxiliary Decoder-Guided Information Concentration method that uses a lightweight auxiliary decoder to enforce content-aware information preservation in the primary latent channels; and 3) a Bitrate-Free Adaptive Textual Conditioning method that derives content-aware textual descriptions from the auxiliary reconstructed image, enabling semantic guidance without bitrate cost.
Abstract (translated)
基于扩散模型的生成图像压缩技术在超低比特率下实现逼真重建方面展现了巨大的潜力。要充分发挥这种潜力，关键在于使整个压缩过程具备内容自适应性，确保编码器表示与解码器先验能够根据输入图像的语义和结构特性动态调整。然而，现有的方法存在三个主要限制，这些限制阻碍了有效的内容适应： 1. **同向量化**：现有方法采用统一的量化步长，未能针对图像内容的空间复杂度变化进行调整，导致与扩散模型噪声依赖先验之间产生不匹配。 2. **信息集中瓶颈**：高维噪声潜在空间和固定输入尺寸的解码器之间的维度不匹配问题，限制了模型在主要通道中自适应地保留关键语义信息的能力。 3. **文本条件策略不足**：现有的文本引导策略要么需要较高的文本比特率开销，要么依赖于通用且无特定内容感知性的提示词，因此无法有效提供适应性语义指导。 为克服这些限制，我们提出了一种基于扩散模型的图像编码解码器框架，并引入了以下三项技术创新： 1. **不确定性导向自适应量化**：通过学习空间不确定度图来动态调整量化失真与内容特性之间的匹配程度。 2. **辅助解码器引导信息集中方法**：利用一个轻量级的辅助解码器，强制在主要潜在通道中保持以内容为中心的信息保留。 3. **无比特率成本自适应文本条件策略**：从辅助重建图像中提取出内容感知性的描述性文本，从而能够在不增加比特率的情况下提供有效的语义指导。
URL
https://arxiv.org/abs/2602.21591
