Abstract
Super-resolution (SR) techniques can enhance the spatial resolution of remote sensing images by utilizing low-resolution (LR) images to reconstruct high-resolution (HR) images, enabling more efficient large-scale earth observation applications. While single-image super-resolution (SISR) methods have shown progress, reference-based super-resolution (RefSR) offers superior performance by incorporating historical HR images alongside current LR observations. However, existing RefSR methods struggle with real-world complexities, such as cross-sensor resolution gap and significant land cover changes, often leading to under-generation or over-reliance on reference image. To address these challenges, we propose CRefDiff, a novel controllable reference-based diffusion model for real-world remote sensing image SR. To address the under-generation problem, CRefDiff is built upon the pretrained Stable Diffusion model, leveraging its powerful generative prior to produce accurate structures and textures. To mitigate over-reliance on the reference, we introduce a dual-branch fusion mechanism that adaptively integrates both local and global information from the reference image. Moreover, this novel dual-branch design enables reference strength control during inference, enhancing interactivity and flexibility of the model. Finally, a strategy named Better Start is proposed to significantly reduce the number of denoising steps, thereby accelerating the inference process. To support further research, we introduce Real-RefRSSRD, a new real-world RefSR dataset for remote sensing images, consisting of HR NAIP and LR Sentinel-2 image pairs with diverse land cover changes and significant temporal gaps. Extensive experiments on Real-RefRSSRD show that CRefDiff achieves state-of-the-art performance across various metrics and improves downstream tasks such as scene classification and semantic segmentation.
Abstract (translated)
超分辨率（SR）技术可以通过利用低分辨率（LR）图像来重建高分辨率（HR）图像，从而提高遥感图像的空间分辨率，并使大规模地球观测应用更加高效。虽然单幅图像超分辨率（SISR）方法已经取得了一些进展，但参考基于的超分辨率（RefSR）通过结合历史上的HR图像和当前的LR观察数据提供了更优越的表现。然而，现有的RefSR方法在处理现实世界的复杂性时面临挑战，例如跨传感器分辨率差异和显著的土地覆盖变化，这些问题往往导致生成不足或过度依赖参考图像。为了解决这些难题，我们提出了CRefDiff，这是一种新型可控参考扩散模型，专门用于遥感图像的真实世界超分辨率任务。 为了应对生成不足的问题，CRefDiff基于预训练的Stable Diffusion模型构建，利用其强大的生成先验来产生准确的结构和纹理信息。为了解决过度依赖参考图像的问题，我们引入了一种双分支融合机制，该机制能够自适应地整合参考图像中的局部和全局信息。此外，这种新颖的双分支设计允许在推理过程中控制参考强度，从而增强模型的交互性和灵活性。最后，提出了名为“Better Start”的策略来大幅减少去噪步骤的数量，从而加速推理过程。 为了支持进一步的研究，我们引入了Real-RefRSSRD，这是一个新的真实世界RefSR数据集，专门用于遥感图像，它包含HR NAIP和LR Sentinel-2图像对，这些图像对具有多样化的土地覆盖变化和显著的时间间隔。在Real-RefRSSRD上的广泛实验表明，CRefDiff在各种指标上达到了最先进的性能，并且提高了下游任务（如场景分类和语义分割）的性能。
URL
https://arxiv.org/abs/2506.23801
