Super_Resolution
-
Super-Resolved Canopy Height Mapping from Sentinel-2 Time Series Using LiDAR HD Reference Data across Metropolitan France
2025-12-12 12:49:16Ekaterina Kalinicheva, Florian Helen, St\'ephane Mermoz, Florian Mouret, Milena PlanellsAbstract
Fine-scale forest monitoring is essential for understanding canopy structure and its dynamics, which are key indicators of carbon stocks, biodiversity, and forest health. Deep learning is particularly effective for this task, as it integrates spectral, temporal, and spatial signals that jointly reflect the canopy structure. To address this need, we introduce THREASURE-Net, a novel end-to-end framework for Tree Height Regression And Super-Resolution. The model is trained on Sentinel-2 time series using reference height metrics derived from LiDAR HD data at multiple spatial resolutions over Metropolitan France to produce annual height maps. We evaluate three model variants, producing tree-height predictions at 2.5 m, 5 m, and 10 m resolution. THREASURE-Net does not rely on any pretrained model nor on reference very high resolution optical imagery to train its super-resolution module; instead, it learns solely from LiDAR-derived height information. Our approach outperforms existing state-of-the-art methods based on Sentinel data and is competitive with methods based on very high resolution imagery. It can be deployed to generate high-precision annual canopy-height maps, achieving mean absolute errors of 2.62 m, 2.72 m, and 2.88 m at 2.5 m, 5 m, and 10 m resolution, respectively. These results highlight the potential of THREASURE-Net for scalable and cost-effective structural monitoring of temperate forests using only freely available satellite data. The source code for THREASURE-Net is available at: this https URL.
Abstract (translated)
精细尺度的森林监测对于理解冠层结构及其动态变化至关重要，这些是衡量碳储量、生物多样性和森林健康状况的关键指标。深度学习在此任务中特别有效，因为它能够整合光谱、时间序列和空间信号，这些信号共同反映了冠层结构的情况。为了满足这一需求，我们引入了THREASURE-Net，这是一种新颖的端到端框架，用于树木高度回归及超分辨率生成。该模型利用Sentinel-2卫星的时间序列数据进行训练，并结合来自法国大都会地区多尺度空间分辨率LiDAR高精度数据所推导出的高度指标来生成年度高度图谱。我们评估了三种模型变体，在2.5米、5米和10米的分辨率下分别产生了树木高度预测。THREASURE-Net不依赖任何预训练模型，也不需要非常高的分辨率光学图像来训练其超分辨率模块；相反，它仅从LiDAR衍生的高度信息中学习。 我们的方法在基于Sentinel数据的方法中超越了现有最先进的技术，并且与基于非常高分辨率影像的方法具有竞争力。它可以部署以生成高精度的年度冠层高度图谱，在2.5米、5米和10米的分辨率下分别实现了平均绝对误差为2.62米、2.72米和2.88米的结果。这些结果突显了THREASURE-Net利用仅限于免费获取的卫星数据就具备对温带森林进行可扩展且成本效益高的结构监测的巨大潜力。 THREASURE-Net的源代码可以在以下链接找到：this https URL。
URL
https://arxiv.org/abs/2512.11524
PDF
-
Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding
2025-12-11 11:27:25Yuchen Feng, Zhenyu Zhang, Naibin Gu, Yilong Chen, Peng Fu, Zheng Lin, Shuohuan Wang, Yu Sun, Hua Wu, Weiping Wang, Haifeng WangAbstract
Multimodal large language models (MLLMs) have achieved remarkable progress on various vision-language tasks, yet their visual perception remains limited. Humans, in comparison, perceive complex scenes efficiently by dynamically scanning and focusing on salient regions in a sequential "blink-like" process. Motivated by this strategy, we first investigate whether MLLMs exhibit similar behavior. Our pilot analysis reveals that MLLMs naturally attend to different visual regions across layers and that selectively allocating more computation to salient tokens can enhance visual perception. Building on this insight, we propose Blink, a dynamic visual token resolution framework that emulates the human-inspired process within a single forward pass. Specifically, Blink includes two modules: saliency-guided scanning and dynamic token resolution. It first estimates the saliency of visual tokens in each layer based on the attention map, and extends important tokens through a plug-and-play token super-resolution (TokenSR) module. In the next layer, it drops the extended tokens when they lose focus. This dynamic mechanism balances broad exploration and fine-grained focus, thereby enhancing visual perception adaptively and efficiently. Extensive experiments validate Blink, demonstrating its effectiveness in enhancing visual perception and multimodal understanding.
Abstract (translated)
多模态大型语言模型（MLLM）在各种视觉-语言任务上取得了显著进展，但其视觉感知能力仍然有限。相比之下，人类通过动态扫描和聚焦于场景中的重要区域，在类似“眨眼”的序列过程中高效地处理复杂场景。受到这一策略的启发，我们首先研究了MLLM是否表现出类似的特性。初步分析显示，MLLM在不同层自然关注不同的视觉区域，并且选择性地向重要的标记分配更多计算资源可以增强其视觉感知能力。基于这一见解，我们提出了一种名为Blink的动态视觉标记解析框架，在单次前向传递中模仿了人类启发的过程。具体而言，Blink包括两个模块：基于显著性的扫描和动态令牌解析。它首先根据注意力图估计每一层中视觉令牌的重要性，并通过即插即用令牌超分辨率（TokenSR）模块扩展重要令牌。在下一层中，当这些扩展的令牌失去焦点时，它们会被丢弃。这种动态机制平衡了广泛的探索与精细聚焦之间的关系，从而能够适应性地、高效地增强视觉感知能力。大量的实验验证了Blink的有效性，证明其可以显著提高视觉感知和多模态理解的能力。
URL
https://arxiv.org/abs/2512.10548
PDF
-
Zero-shot Adaptation of Stable Diffusion via Plug-in Hierarchical Degradation Representation for Real-World Super-Resolution
2025-12-11 06:45:28Yi-Cheng Liao, Shyang-En Weng, Yu-Syuan Xu, Chi-Wei Hsiao, Wei-Chen Chiu, Ching-Chun HuangAbstract
Real-World Image Super-Resolution (Real-ISR) aims to recover high-quality images from low-quality inputs degraded by unknown and complex real-world factors. Real-world scenarios involve diverse and coupled degradations, making it necessary to provide diffusion models with richer and more informative guidance. However, existing methods often assume known degradation severity and rely on CLIP text encoders that cannot capture numerical severity, limiting their generalization ability. To address this, we propose \textbf{HD-CLIP} (\textbf{H}ierarchical \textbf{D}egradation CLIP), which decomposes a low-quality image into a semantic embedding and an ordinal degradation embedding that captures ordered relationships and allows interpolation across unseen levels. Furthermore, we integrated it into diffusion models via classifier-free guidance (CFG) and proposed classifier-free projection guidance (CFPG). HD-CLIP leverages semantic cues to guide generative restoration while using degradation cues to suppress undesired hallucinations and artifacts. As a \textbf{plug-and-play module}, HD-CLIP can be seamlessly integrated into various super-resolution frameworks without training, significantly improving detail fidelity and perceptual realism across diverse real-world datasets.
Abstract (translated)
现实世界图像超分辨率（Real-ISR）旨在从由未知且复杂的现实因素降级的低质量输入中恢复高质量的图像。在实际场景中，存在多样化的和耦合的退化情况，这就需要提供给扩散模型更丰富和更有信息量的指导。然而，现有的方法往往假设已知的退化程度，并依赖于CLIP文本编码器（这些编码器不能捕捉数值上的严重程度），这限制了它们的泛化能力。 为了应对这一挑战，我们提出了一种名为**HD-CLIP**（**H**ierarchical **D**egradation CLIP）的方法。该方法将低质量图像分解为一个语义嵌入和一个序数退化嵌入，后者可以捕获有序关系，并允许在未见过的水平之间进行插值。此外，我们将这种方法整合到扩散模型中，通过无分类器指导（CFG）提出了一种新的引导方式——无分类器投影指导（CFPG）。HD-CLIP利用语义线索来引导生成性恢复过程，同时使用退化线索来抑制不需要的幻觉和伪影。 作为**即插即用模块**，HD-CLIP可以无缝地集成到各种超分辨率框架中，而无需进行额外训练。这显著提高了在多种现实世界数据集上的细节保真度和感知逼真性。
URL
https://arxiv.org/abs/2512.10340
PDF
-
A Dual-Domain Convolutional Network for Hyperspectral Single-Image Super-Resolution
2025-12-10 11:35:38Murat Karayaka, Usman Muhammad, Jorma Laaksonen, Md Ziaul Hoque, Tapio Sepp\"anenAbstract
This study presents a lightweight dual-domain super-resolution network (DDSRNet) that combines Spatial-Net with the discrete wavelet transform (DWT). Specifically, our proposed model comprises three main components: (1) a shallow feature extraction module, termed Spatial-Net, which performs residual learning and bilinear interpolation; (2) a low-frequency enhancement branch based on the DWT that refines coarse image structures; and (3) a shared high-frequency refinement branch that simultaneously enhances the LH (horizontal), HL (vertical), and HH (diagonal) wavelet subbands using a single CNN with shared weights. As a result, the DWT enables subband decomposition, while the inverse DWT reconstructs the final high-resolution output. By doing so, the integration of spatial- and frequency-domain learning enables DDSRNet to achieve highly competitive performance with low computational cost on three hyperspectral image datasets, demonstrating its effectiveness for hyperspectral image super-resolution.
Abstract (translated)
这项研究提出了一种轻量级的双域超分辨率网络（DDSRNet），结合了空间网络（Spatial-Net）和离散小波变换（DWT）。具体来说，我们提出的模型包括三个主要组成部分：(1) 一个浅层特征提取模块，称为空间网络（Spatial-Net），执行残差学习和双线性插值；(2) 基于DWT的低频增强分支，用于优化粗略图像结构；以及 (3) 共享高频细化分支，在水平（LH）、垂直（HL）和对角线（HH）小波子带中同时进行精细处理，采用单个具有共享权重的CNN实现。因此，DWT实现了子带分解的功能，而逆DWT则重建最终的高分辨率输出。 通过这种方式，空间域与频率域学习的结合使DDSRNet能够在三个高光谱图像数据集上以低计算成本达到高度竞争性的性能，展示了其在高光谱图像超分辨率任务中的有效性。
URL
https://arxiv.org/abs/2512.09546
PDF
-
FunPhase: A Periodic Functional Autoencoder for Motion Generation via Phase Manifolds
2025-12-10 08:46:53Marco Pegoraro, Evan Atherton, Bruno Roy, Aliasghar Khani, Arianna RampiniAbstract
Learning natural body motion remains challenging due to the strong coupling between spatial geometry and temporal dynamics. Embedding motion in phase manifolds, latent spaces that capture local periodicity, has proven effective for motion prediction; however, existing approaches lack scalability and remain confined to specific settings. We introduce FunPhase, a functional periodic autoencoder that learns a phase manifold for motion and replaces discrete temporal decoding with a function-space formulation, enabling smooth trajectories that can be sampled at arbitrary temporal resolutions. FunPhase supports downstream tasks such as super-resolution and partial-body motion completion, generalizes across skeletons and datasets, and unifies motion prediction and generation within a single interpretable manifold. Our model achieves substantially lower reconstruction error than prior periodic autoencoder baselines while enabling a broader range of applications and performing on par with state-of-the-art motion generation methods.
Abstract (translated)
学习自然身体动作仍然具有挑战性，因为空间几何和时间动态之间存在强烈的耦合。将运动嵌入到相位流形中（即捕获局部周期性的潜在空间）已被证明对于运动预测非常有效；然而，现有的方法缺乏可扩展性，并且仍局限于特定的场景设置内。我们引入了FunPhase，这是一种功能性周期自动编码器，它为动作学习了一个相位流形，并用函数空间形式代替了离散的时间解码，从而实现了可以在任意时间分辨率下采样的平滑轨迹。FunPhase支持下游任务，如超分辨率和部分身体运动完成，能够在不同的骨骼结构和数据集中泛化，并且将运动预测与生成统一在一个单一的可解释流形内。我们的模型在重建误差方面显著低于先前的周期自动编码器基准线，同时实现了更广泛的应用范围，并且其性能与最先进的运动生成方法相当。
URL
https://arxiv.org/abs/2512.09423
PDF
-
SuperF: Neural Implicit Fields for Multi-Image Super-Resolution
2025-12-09 20:57:25Sander Riis{\o}en Jyhne, Christian Igel, Morten Goodwin, Per-Arne Andersen, Serge Belongie, Nico LangAbstract
High-resolution imagery is often hindered by limitations in sensor technology, atmospheric conditions, and costs. Such challenges occur in satellite remote sensing, but also with handheld cameras, such as our smartphones. Hence, super-resolution aims to enhance the image resolution algorithmically. Since single-image super-resolution requires solving an inverse problem, such methods must exploit strong priors, e.g. learned from high-resolution training data, or be constrained by auxiliary data, e.g. by a high-resolution guide from another modality. While qualitatively pleasing, such approaches often lead to "hallucinated" structures that do not match reality. In contrast, multi-image super-resolution (MISR) aims to improve the (optical) resolution by constraining the super-resolution process with multiple views taken with sub-pixel shifts. Here, we propose SuperF, a test-time optimization approach for MISR that leverages coordinate-based neural networks, also called neural fields. Their ability to represent continuous signals with an implicit neural representation (INR) makes them an ideal fit for the MISR task. The key characteristic of our approach is to share an INR for multiple shifted low-resolution frames and to jointly optimize the frame alignment with the INR. Our approach advances related INR baselines, adopted from burst fusion for layer separation, by directly parameterizing the sub-pixel alignment as optimizable affine transformation parameters and by optimizing via a super-sampled coordinate grid that corresponds to the output resolution. Our experiments yield compelling results on simulated bursts of satellite imagery and ground-level images from handheld cameras, with upsampling factors of up to 8. A key advantage of SuperF is that this approach does not rely on any high-resolution training data.
Abstract (translated)
高分辨率成像经常受到传感器技术、大气条件和成本的限制。这些挑战不仅存在于卫星遥感中，也出现在手持设备如智能手机摄像头拍摄的照片上。因此，超分辨率旨在通过算法增强图像的清晰度。单幅图像的超分辨率需要解决一个逆向问题，这类方法必须利用强大的先验信息，例如从高分辨率训练数据中学到的信息，或者受到辅助数据（例如其他模态提供的高分辨率指南）的限制。尽管这些方法在视觉上令人满意，但它们往往会产生与现实不符的“幻觉”结构。相比之下，多图像超分辨率（MISR）的目标是通过使用具有亚像素偏移的多个视图来提高（光学）分辨率。在这里，我们提出了一种名为SuperF的方法，这是一种针对MISR的测试时优化方法，它利用了坐标基神经网络或称为神经场的模型。由于它们能够用隐式神经表示（INR）连续信号的能力，使其非常适合于MISR任务。我们的方法的关键特征是为多个偏移低分辨率帧共享一个INR，并联合优化帧对齐与INR的过程。相较于从层分离的突发融合中采用的相关INR基准线，SuperF直接参数化亚像素对准作为可优化仿射变换参数，并通过对应于输出分辨率的超采样坐标网格进行优化。我们在模拟卫星图像和手持相机拍摄的地面级图像的爆发数据上进行了实验，实现了高达8倍的上采样率。SuperF的一个关键优势是这种方法不依赖任何高分辨率训练数据。
URL
https://arxiv.org/abs/2512.09115
PDF
-
FlowSteer: Conditioning Flow Field for Consistent Image Restoration
2025-12-09 00:09:21Tharindu Wickremasinghe, Chenyang Qi, Harshana Weligampola, Zhengzhong Tu, Stanley H. ChanAbstract
Flow-based text-to-image (T2I) models excel at prompt-driven image generation, but falter on Image Restoration (IR), often "drifting away" from being faithful to the measurement. Prior work mitigate this drift with data-specific flows or task-specific adapters that are computationally heavy and not scalable across tasks. This raises the question "Can't we efficiently manipulate the existing generative capabilities of a flow model?" To this end, we introduce FlowSteer (FS), an operator-aware conditioning scheme that injects measurement priors along the sampling path,coupling a frozed flow's implicit guidance with explicit measurement constraints. Across super-resolution, deblurring, denoising, and colorization, FS improves measurement consistency and identity preservation in a strictly zero-shot setting-no retrained models, no adapters. We show how the nature of flow models and their sensitivities to noise inform the design of such a scheduler. FlowSteer, although simple, achieves a higher fidelity of reconstructed images, while leveraging the rich generative priors of flow models.
Abstract (translated)
基于流的文本到图像（T2I）模型在指令驱动的图像生成方面表现出色，但在图像修复（IR）任务上表现不佳，往往偏离了对测量值的忠实度。先前的工作通过使用特定数据的流或特定任务的适配器来减轻这种漂移现象，但这些方法计算成本高且无法跨任务扩展。这引发了这样一个问题：“我们能否有效地操纵现有生成模型的能力？”为此，我们引入了一种名为FlowSteer（FS）的操作感知条件方案，该方案在采样路径上注入测量先验信息，并结合冻结流的隐式指导与明确的测量约束。 在超分辨率、去模糊化、降噪和色彩处理等任务中，FlowSteer在严格零样本设置下提升了图像的一致性和身份保持性——无需重新训练模型或适配器。我们展示了流模型的本质及其对噪声的敏感度是如何影响调度设计的。尽管简单，FlowSteer通过利用流模型丰富的生成先验信息实现了更高的重建图像保真度。 总结来说，这项工作提出了一个简洁而高效的方法，可以增强基于流的模型在图像修复任务中的性能，并且无需额外训练或复杂适配器的支持，在零样本设置下实现显著改进。
URL
https://arxiv.org/abs/2512.08125
PDF
-
Evaluating and Preserving High-level Fidelity in Super-Resolution
2025-12-07 22:53:34Josep M. Rocafort, Shaolin Su, Javier Vazquez-Corral, Alexandra Gomez-VillaAbstract
Recent image Super-Resolution (SR) models are achieving impressive effects in reconstructing details and delivering visually pleasant outputs. However, the overpowering generative ability can sometimes hallucinate and thus change the image content despite gaining high visual quality. This type of high-level change can be easily identified by humans yet not well-studied in existing low-level image quality metrics. In this paper, we establish the importance of measuring high-level fidelity for SR models as a complementary criterion to reveal the reliability of generative SR models. We construct the first annotated dataset with fidelity scores from different SR models, and evaluate how state-of-the-art (SOTA) SR models actually perform in preserving high-level fidelity. Based on the dataset, we then analyze how existing image quality metrics correlate with fidelity measurement, and further show that this high-level task can be better addressed by foundation models. Finally, by fine-tuning SR models based on our fidelity feedback, we show that both semantic fidelity and perceptual quality can be improved, demonstrating the potential value of our proposed criteria, both in model evaluation and optimization. We will release the dataset, code, and models upon acceptance.
Abstract (translated)
最近的图像超分辨率（SR）模型在重建细节和提供视觉上令人愉悦的结果方面取得了显著效果。然而，强大的生成能力有时会导致幻觉现象，并因此改变图像内容，尽管这些变化可能使视觉质量提高。这种高层次的变化虽然容易被人类识别，但现有的低层次图像质量度量却未能充分研究。本文确立了评估SR模型的高层次保真度的重要性，作为衡量生成型超分辨率模型可靠性的补充标准。我们构建了一个带有来自不同SR模型的保真度评分的数据集，并评估最先进的（SOTA）SR模型在保持高层次保真度方面的实际表现。基于该数据集，我们分析现有图像质量指标与保真度测量之间的相关性，并进一步表明基础模型可以更好地解决这一高级任务。最后，通过根据我们的保真度反馈微调SR模型，我们展示了语义保真度和感知质量都可以得到改善，证明了所提出的评价标准在模型评估和优化中的潜在价值。我们将接受后发布该数据集、代码及模型。
URL
https://arxiv.org/abs/2512.07037
PDF
-
Masked Autoencoder Pretraining on Strong-Lensing Images for Joint Dark-Matter Model Classification and Super-Resolution
2025-12-07 03:25:19Achmad Ardani Prasha, Clavino Ourizqi Rachmadi, Muhamad Fauzan Ibnu Syahlan, Naufal Rahfi Anugerah, Nanda Garin Raditya, Putri Amelia, Sabrina Laila Mutiara, Hilman Syachr RamadhanAbstract
Strong gravitational lensing can reveal the influence of dark-matter substructure in galaxies, but analyzing these effects from noisy, low-resolution images poses a significant challenge. In this work, we propose a masked autoencoder (MAE) pretraining strategy on simulated strong-lensing images from the DeepLense ML4SCI benchmark to learn generalizable representations for two downstream tasks: (i) classifying the underlying dark matter model (cold dark matter, axion-like, or no substructure) and (ii) enhancing low-resolution lensed images via super-resolution. We pretrain a Vision Transformer encoder using a masked image modeling objective, then fine-tune the encoder separately for each task. Our results show that MAE pretraining, when combined with appropriate mask ratio tuning, yields a shared encoder that matches or exceeds a ViT trained from scratch. Specifically, at a 90% mask ratio, the fine-tuned classifier achieves macro AUC of 0.968 and accuracy of 88.65%, compared to the scratch baseline (AUC 0.957, accuracy 82.46%). For super-resolution (16x16 to 64x64), the MAE-pretrained model reconstructs images with PSNR ~33 dB and SSIM 0.961, modestly improving over scratch training. We ablate the MAE mask ratio, revealing a consistent trade-off: higher mask ratios improve classification but slightly degrade reconstruction fidelity. Our findings demonstrate that MAE pretraining on physics-rich simulations provides a flexible, reusable encoder for multiple strong-lensing analysis tasks.
Abstract (translated)
强引力透镜效应可以揭示星系中暗物质亚结构的影响，但从嘈杂、低分辨率的图像中分析这些影响是一个重大的挑战。在本工作中，我们提出了一种使用DeepLense ML4SCI基准中的模拟强引力透镜图像对掩码自动编码器（MAE）进行预训练的战略，以学习适用于两个下游任务的一般化表示：(i) 分类底层暗物质模型（冷暗物质、类似轴子或无亚结构），以及(ii) 通过超分辨率增强低分辨率的透镜图像。我们使用带掩码图像建模目标对视觉变换器编码器进行预训练，然后分别针对每个任务微调该编码器。 实验结果表明，当与适当的掩码比率调整相结合时，MAE 预训练可以生成一个共享编码器，其性能可匹敌或超越从头开始训练的ViT。特别是在90% 的掩码比率下，经过微调的分类器实现了宏AUC值为0.968和准确性为88.65%，相比之下，从零开始的基准模型（宏AUC 0.957，准确性 82.46%)。 对于超分辨率任务（从16x16到64x64），MAE预训练模型重建图像时获得了约33 dB 的PSNR值和SSIM为0.961的得分，这比从零开始训练略有提升。通过对MAE掩码比率进行消融分析，我们发现了一个一致的权衡：更高的掩码比率可以提高分类性能，但稍微降低了重建保真度。 我们的研究结果表明，通过在富含物理特性的模拟数据上对MAE进行预训练，可以获得一个灵活且可重用的编码器，用于多种强引力透镜分析任务。
URL
https://arxiv.org/abs/2512.06642
PDF
-
China Regional 3km Downscaling Based on Residual Corrective Diffusion Model
2025-12-05 02:27:08Honglu Sun, Hao Jing, Zhixiang Dai, Sa Xiao, Wei Xue, Jian Sun, Qifeng LuAbstract
A fundamental challenge in numerical weather prediction is to efficiently produce high-resolution forecasts. A common solution is applying downscaling methods, which include dynamical downscaling and statistical downscaling, to the outputs of global models. This work focuses on statistical downscaling, which establishes statistical relationships between low-resolution and high-resolution historical data using statistical models. Deep learning has emerged as a powerful tool for this task, giving rise to various high-performance super-resolution models, which can be directly applied for downscaling, such as diffusion models and Generative Adversarial Networks. This work relies on a diffusion-based downscaling framework named CorrDiff. In contrast to the original work of CorrDiff, the region considered in this work is nearly 20 times larger, and we not only consider surface variables as in the original work, but also encounter high-level variables (six pressure levels) as target downscaling variables. In addition, a global residual connection is added to improve accuracy. In order to generate the 3km forecasts for the China region, we apply our trained models to the 25km global grid forecasts of CMA-GFS, an operational global model of the China Meteorological Administration (CMA), and SFF, a data-driven deep learning-based weather model developed from Spherical Fourier Neural Operators (SFNO). CMA-MESO, a high-resolution regional model, is chosen as the baseline model. The experimental results demonstrate that the forecasts downscaled by our method generally outperform the direct forecasts of CMA-MESO in terms of MAE for the target variables. Our forecasts of radar composite reflectivity show that CorrDiff, as a generative model, can generate fine-scale details that lead to more realistic predictions compared to the corresponding deterministic regression models.
Abstract (translated)
数值天气预报中的一个基本挑战是高效地生成高分辨率的预测。一种常见的解决方案是采用下采样方法，包括动力学下采样和统计下采样，来处理全球模型输出的数据。本研究专注于统计下采样，即利用统计模型在低分辨率与高分辨率历史数据之间建立统计关系的方法。深度学习作为一种强大的工具，在此任务中得到广泛应用，并催生了各种高性能的超分辨率模型，如扩散模型和生成对抗网络（GAN），可以直接应用于下采样过程。 这项工作基于一个名为CorrDiff的扩散基础下采样框架。与原始的CorrDiff相比，本研究涉及的区域几乎是其20倍大，并且我们不仅考虑了表面变量，还处理到了六个气压层的高度变量作为目标下采样变量。此外，在此模型中添加了一个全局残差连接以提高精度。 为了生成中国地区的3公里预报，我们将训练好的模型应用于中国的气象局（CMA）操作全球模式CMA-GFS和从球面傅立叶神经算子(SFNO)发展的数据驱动深度学习天气模型SFF所给出的25公里全球网格预报。高分辨率区域模型CMA-MESO被选作基线模型。 实验结果表明，通过我们方法下采样的预测通常在目标变量方面（以平均绝对误差(MAE)衡量）优于直接由CMA-MESO生成的预测。我们的雷达综合反射率预报显示，作为生成模型的CorrDiff能够产生更精细的细节，从而比相应的确定性回归模型提供更加真实的预测结果。
URL
https://arxiv.org/abs/2512.05377
PDF
-
CARD: Correlation Aware Restoration with Diffusion
2025-12-04 21:46:43Niki Nezakati, Arnab Ghosh, Amit Roy-Chowdhury, Vishwanath SaragadamAbstract
Denoising diffusion models have achieved state-of-the-art performance in image restoration by modeling the process as sequential denoising steps. However, most approaches assume independent and identically distributed (i.i.d.) Gaussian noise, while real-world sensors often exhibit spatially correlated noise due to readout mechanisms, limiting their practical effectiveness. We introduce Correlation Aware Restoration with Diffusion (CARD), a training-free extension of DDRM that explicitly handles correlated Gaussian noise. CARD first whitens the noisy observation, which converts the noise into an i.i.d. form. Then, the diffusion restoration steps are replaced with noise-whitened updates, which inherits DDRM's closed-form sampling efficiency while now being able to handle correlated noise. To emphasize the importance of addressing correlated noise, we contribute CIN-D, a novel correlated noise dataset captured across diverse illumination conditions to evaluate restoration methods on real rolling-shutter sensor noise. This dataset fills a critical gap in the literature for experimental evaluation with real-world correlated noise. Experiments on standard benchmarks with synthetic correlated noise and on CIN-D demonstrate that CARD consistently outperforms existing methods across denoising, deblurring, and super-resolution tasks.
Abstract (translated)
去噪扩散模型通过将图像恢复过程建模为一系列逐步去噪步骤，已经在图像恢复方面取得了最先进的性能。然而，大多数方法假设噪声是独立同分布（i.i.d.）高斯噪声，而实际情况中，由于读出机制的存在，真实世界的传感器常常会产生空间相关的噪声，这限制了现有模型的实际效果。 我们提出了CARD（Correlation Aware Restoration with Diffusion），这是一种无训练扩展的DDRM（Denoising Diffusion Restoration Model）方法，可以显式处理相关高斯噪声。CARD首先对带噪观察进行白化处理，将噪声转换为i.i.d.形式。然后，用去噪白化的更新替换原有的扩散恢复步骤，在继承DDRM闭合形式采样效率的同时，现在也可以处理相关的噪声。 为了强调解决相关噪声的重要性，我们贡献了一种新的数据集CIN-D（Correlated Illumination Noise Dataset），该数据集在不同的光照条件下捕捉到了真实滚动快门传感器噪声。这个数据集填补了现有文献中使用现实世界相关噪声进行实验评估的重要空白。 标准基准测试和合成的相关噪声以及CIN-D上的实验证明，CARD在去噪、去模糊和超分辨率任务上始终优于现有的方法。
URL
https://arxiv.org/abs/2512.05268
PDF
-
DraCo: Draft as CoT for Text-to-Image Preview and Rare Concept Generation
2025-12-04 18:59:53Dongzhi Jiang, Renrui Zhang, Haodong Li, Zhuofan Zong, Ziyu Guo, Jun He, Claire Guo, Junyan Ye, Rongyao Fang, Weijia Li, Rui Liu, Hongsheng LiAbstract
Recent unified multimodal large language models (MLLMs) have shown impressive capabilities, incorporating chain-of-thought (CoT) reasoning for enhanced text-to-image generation. However, existing approaches remain limited, either treating the model merely as a standalone generator or relying on abstract textual planning. To this end, we propose Draft-as-CoT (DraCo), a novel interleaved reasoning paradigm that fully leverages both textual and visual contents in CoT for better planning and verification. Our method first generates a low-resolution draft image as preview, providing more concrete and structural visual planning and guidance. Then, we employ the model's inherent understanding capability to verify potential semantic misalignments between the draft and input prompt, and performs refinement through selective corrections with super-resolution. In this way, our approach addresses two fundamental challenges: the coarse-grained nature of textual planning and the difficulty in generating rare attribute combinations. To support training, we curate DraCo-240K, aiming to enhance three atomic capabilities spanning general correction, instance manipulation, and layout reorganization. Supported by DraCo-CFG, a specialized classifier-free guidance (CFG) strategy for interleaved reasoning, DraCo achieves a tremendous increase on GenEval (+8%), Imagine-Bench (+0.91), and GenEval++ (+3%), significantly outperforming direct generation and other generation methods empowered by CoT.
Abstract (translated)
最近统一的多模态大型语言模型（MLLMs）展示了令人印象深刻的性能，通过链式思维（CoT）推理增强了文本到图像生成的能力。然而，现有的方法仍然有限，要么仅仅将模型视为独立的生成器，要么依赖于抽象的文字规划。为了解决这些问题，我们提出了Draft-as-CoT (DraCo)，这是一种新的交替推理范例，全面利用了链式思维中的文字和视觉内容以实现更好的计划与验证。 我们的方法首先生成低分辨率的草图图像作为预览，提供更具体且结构化的视觉规划和指导。然后，通过模型内在的理解能力来核实草图与输入提示之间的潜在语义不一致，并通过选择性修正并结合超分辨率技术进行细化处理。这样一来，我们的方法解决了两个根本性的挑战：文本规划的粗粒度性质以及生成罕见属性组合的难度。 为了支持训练，我们整理了DraCo-240K数据集，旨在增强三种基本能力——通用校正、实例操作和布局重组。通过专门用于交替推理的无分类器引导（CFG）策略DraCo-CFG的支持，DraCo在GenEval中提高了8%，在Imagine-Bench中提高了0.91，在GenEval++中提高了3%的成绩，明显优于直接生成和其他受链式思维增强的方法。
URL
https://arxiv.org/abs/2512.05112
PDF
-
LaFiTe: A Generative Latent Field for 3D Native Texturing
2025-12-04 13:33:49Chia-Hao Chen, Zi-Xin Zou, Yan-Pei Cao, Ze Yuan, Guan Luo, Xiaojuan Qi, Ding Liang, Song-Hai Zhang, Yuan-Chen GuoAbstract
Generating high-fidelity, seamless textures directly on 3D surfaces, what we term 3D-native texturing, remains a fundamental open challenge, with the potential to overcome long-standing limitations of UV-based and multi-view projection methods. However, existing native approaches are constrained by the absence of a powerful and versatile latent representation, which severely limits the fidelity and generality of their generated textures. We identify this representation gap as the principal barrier to further progress. We introduce LaFiTe, a framework that addresses this challenge by learning to generate textures as a 3D generative sparse latent color field. At its core, LaFiTe employs a variational autoencoder (VAE) to encode complex surface appearance into a sparse, structured latent space, which is subsequently decoded into a continuous color field. This representation achieves unprecedented fidelity, exceeding state-of-the-art methods by >10 dB PSNR in reconstruction, by effectively disentangling texture appearance from mesh topology and UV parameterization. Building upon this strong representation, a conditional rectified-flow model synthesizes high-quality, coherent textures across diverse styles and geometries. Extensive experiments demonstrate that LaFiTe not only sets a new benchmark for 3D-native texturing but also enables flexible downstream applications such as material synthesis and texture super-resolution, paving the way for the next generation of 3D content creation workflows.
Abstract (translated)
生成高质量、无缝贴图直接在三维表面上进行，我们称之为原生三维纹理化，仍然是一个基本的开放挑战。这种方法有可能克服基于UV映射和多视角投影方法长期以来存在的局限性。然而，现有的原生方法由于缺乏强大且通用的潜在表示而受到限制，这严重制约了它们生成的贴图质量和适用性。我们认为这一表示差距是进一步发展的主要障碍。 为此，我们引入了一种名为LaFiTe（Latent Fidelity Texturing）的新框架，它通过学习将纹理作为三维生成稀疏潜在颜色场来解决这一挑战。LaFiTe的核心采用变分自编码器(VAE)技术，将复杂的表面外观编码进一个稀疏且结构化的潜在空间中，并进一步解码为连续的颜色字段。这种表示方法实现了前所未有的保真度，在重建时超过了当前最佳方法10dB以上的PSNR（峰值信噪比），通过有效地分离纹理外观与网格拓扑和UV参数化，从而提高质量。 在此强大的表示基础上，条件正流模型能够合成出高质量、连贯的贴图，涵盖各种风格和几何形态。一系列广泛的实验表明，LaFiTe不仅为原生三维纹理化设立了新的基准，而且还支持了诸如材质生成和纹理超分辨率等灵活的下游应用的发展，从而开启了下一代三维内容创建工作流程的大门。
URL
https://arxiv.org/abs/2512.04786
PDF
-
OmniScaleSR: Unleashing Scale-Controlled Diffusion Prior for Faithful and Realistic Arbitrary-Scale Image Super-Resolution
2025-12-04 11:50:17Xinning Chai, Zhengxue Cheng, Yuhong Zhang, Hengsheng Zhang, Yingsheng Qin, Yucai Yang, Rong Xie, Li SongAbstract
Arbitrary-scale super-resolution (ASSR) overcomes the limitation of traditional super-resolution (SR) methods that operate only at fixed scales (e.g., 4x), enabling a single model to handle arbitrary magnification. Most existing ASSR approaches rely on implicit neural representation (INR), but its regression-driven feature extraction and aggregation intrinsically limit the ability to synthesize fine details, leading to low realism. Recent diffusion-based realistic image super-resolution (Real-ISR) models leverage powerful pre-trained diffusion priors and show impressive results at the 4x setting. We observe that they can also achieve ASSR because the diffusion prior implicitly adapts to scale by encouraging high-realism generation. However, without explicit scale control, the diffusion process cannot be properly adjusted for different magnification levels, resulting in excessive hallucination or blurry outputs, especially under ultra-high scales. To address these issues, we propose OmniScaleSR, a diffusion-based realistic arbitrary-scale SR framework designed to achieve both high fidelity and high realism. We introduce explicit, diffusion-native scale control mechanisms that work synergistically with implicit scale adaptation, enabling scale-aware and content-aware modulation of the diffusion process. In addition, we incorporate multi-domain fidelity enhancement designs to further improve reconstruction accuracy. Extensive experiments on bicubic degradation benchmarks and real-world datasets show that OmniScaleSR surpasses state-of-the-art methods in both fidelity and perceptual realism, with particularly strong performance at large magnification factors. Code will be released at this https URL.
Abstract (translated)
任意尺度超分辨率（ASSR）克服了传统超分辨率（SR）方法仅在固定尺度下操作的限制，使得单一模型能够处理任意级别的放大。大多数现有的ASSR方法依赖于隐式神经表示（INR），但其基于回归驱动的特征提取和聚合机制内在地限制了合成精细细节的能力，导致现实感较低。最近基于扩散的真实图像超分辨率（Real-ISR）模型利用强大的预训练扩散先验，并在4倍放大设置下表现出令人印象深刻的结果。我们观察到它们也可以实现ASSR，因为扩散先验通过鼓励高现实感生成而隐式地适应尺度变化。然而，在没有显式尺度控制的情况下，扩散过程无法针对不同的放大级别进行适当调整，这会导致过多的幻觉或模糊输出，尤其是在超高尺度下。 为了解决这些问题，我们提出了OmniScaleSR，这是一种基于扩散的真实任意尺度超分辨率框架，旨在同时实现高保真度和高现实感。我们引入了显式的、与扩散本源兼容的尺度控制机制，这些机制与隐式尺度适应协同工作，能够以尺度感知和内容感知的方式调节扩散过程。此外，我们将多域保真度增强设计纳入其中，进一步提高重建精度。 在双三次降级基准测试和现实世界数据集上进行的大量实验表明，OmniScaleSR在保真度和感知真实感方面超越了最先进的方法，在大放大倍数下表现尤为突出。代码将在[此处](此链接应指向代码发布页面)提供。
URL
https://arxiv.org/abs/2512.04699
PDF
-
FMA-Net++: Motion- and Exposure-Aware Real-World Joint Video Super-Resolution and Deblurring
2025-12-04 02:23:52Geunhyuk Youk, Jihyong Oh, Munchurl KimAbstract
Real-world video restoration is plagued by complex degradations from motion coupled with dynamically varying exposure - a key challenge largely overlooked by prior works and a common artifact of auto-exposure or low-light capture. We present FMA-Net++, a framework for joint video super-resolution and deblurring that explicitly models this coupled effect of motion and dynamically varying exposure. FMA-Net++ adopts a sequence-level architecture built from Hierarchical Refinement with Bidirectional Propagation blocks, enabling parallel, long-range temporal modeling. Within each block, an Exposure Time-aware Modulation layer conditions features on per-frame exposure, which in turn drives an exposure-aware Flow-Guided Dynamic Filtering module to infer motion- and exposure-aware degradation kernels. FMA-Net++ decouples degradation learning from restoration: the former predicts exposure- and motion-aware priors to guide the latter, improving both accuracy and efficiency. To evaluate under realistic capture conditions, we introduce REDS-ME (multi-exposure) and REDS-RE (random-exposure) benchmarks. Trained solely on synthetic data, FMA-Net++ achieves state-of-the-art accuracy and temporal consistency on our new benchmarks and GoPro, outperforming recent methods in both restoration quality and inference speed, and generalizes well to challenging real-world videos.
Abstract (translated)
现实世界的视频修复面临由于运动与动态变化曝光相互作用而导致的复杂退化问题——这一挑战被之前的大部分工作忽视，且通常是自动曝光或低光捕捉下的常见缺陷。我们提出了FMA-Net++框架，该框架用于联合进行视频超分辨率和去模糊处理，并明确地建模了运动与动态变化曝光相结合的效果。 FMA-Net++采用了一种基于双向传播块的层级细化序列级架构，这使得长期时序并行模型得以实现。在每个模块中，一种带有曝光时间感知调节层（Exposure Time-aware Modulation layer）的结构根据每一帧的曝光条件来调整特征。这一机制驱动了一个带曝光意识的流导向动态滤波模块（exposure-aware Flow-Guided Dynamic Filtering module），用于推断运动和曝光相关的退化内核。 FMA-Net++将退化的学习与修复过程解耦：前者预测带有曝光和运动感知的先验信息，以指导后者进行更准确且高效的恢复。为了评估在现实捕捉条件下的性能，我们引入了REDs-ME（多曝光）和REDs-RE（随机曝光）基准测试。 仅使用合成数据训练后，FMA-Net++在我们的新基准以及GoPro上实现了最新的精度与时序一致性表现，在修复质量和推理速度方面优于近期方法，并且能很好地推广到具有挑战性的现实视频中。
URL
https://arxiv.org/abs/2512.04390
PDF
-
Learning Single-Image Super-Resolution in the JPEG Compressed Domain
2025-12-03 21:51:57Sruthi Srinivasan, Elham Shakibapour, Rajy Rawther, Mehdi SaeediAbstract
Deep learning models have grown increasingly complex, with input data sizes scaling accordingly. Despite substantial advances in specialized deep learning hardware, data loading continues to be a major bottleneck that limits training and inference speed. To address this challenge, we propose training models directly on encoded JPEG features, reducing the computational overhead associated with full JPEG decoding and significantly improving data loading efficiency. While prior works have focused on recognition tasks, we investigate the effectiveness of this approach for the restoration task of single-image super-resolution (SISR). We present a lightweight super-resolution pipeline that operates on JPEG discrete cosine transform (DCT) coefficients in the frequency domain. Our pipeline achieves a 2.6x speedup in data loading and a 2.5x speedup in training, while preserving visual quality comparable to standard SISR approaches.
Abstract (translated)
深度学习模型变得越来越复杂，输入数据的大小也随之增加。尽管在专门用于深度学习的硬件方面取得了重大进展，但数据加载仍然是限制训练和推理速度的主要瓶颈。为了解决这一挑战，我们提出了一种直接在编码JPEG特征上训练模型的方法，这减少了完全解码JPEG所带来的计算开销，并显著提高了数据加载效率。虽然先前的研究主要集中在识别任务上，但我们研究了这种方法对单图像超分辨率（SISR）恢复任务的有效性。我们介绍了一个轻量级的超分辨率管道，该管道操作于JPEG离散余弦变换（DCT）系数在频域中。我们的管道实现了数据加载速度提高2.6倍和训练速度提高2.5倍，并且保持了与标准SISR方法相当的视觉质量。
URL
https://arxiv.org/abs/2512.04284
PDF
-
Plug-and-Play Image Restoration with Flow Matching: A Continuous Viewpoint
2025-12-03 21:50:36Fan Jia, Yuhao Huang, Shih-Hsin Wang, Cristina Garcia-Cardona, Andrea L. Bertozzi, Bao WangAbstract
Flow matching-based generative models have been integrated into the plug-and-play image restoration framework, and the resulting plug-and-play flow matching (PnP-Flow) model has achieved some remarkable empirical success for image restoration. However, the theoretical understanding of PnP-Flow lags its empirical success. In this paper, we derive a continuous limit for PnP-Flow, resulting in a stochastic differential equation (SDE) surrogate model of PnP-Flow. The SDE model provides two particular insights to improve PnP-Flow for image restoration: (1) It enables us to quantify the error for image restoration, informing us to improve step scheduling and regularize the Lipschitz constant of the neural network-parameterized vector field for error reduction. (2) It informs us to accelerate off-the-shelf PnP-Flow models via extrapolation, resulting in a rescaled version of the proposed SDE model. We validate the efficacy of the SDE-informed improved PnP-Flow using several benchmark tasks, including image denoising, deblurring, super-resolution, and inpainting. Numerical results show that our method significantly outperforms the baseline PnP-Flow and other state-of-the-art approaches, achieving superior performance across evaluation metrics.
Abstract (translated)
基于流匹配的生成模型已被集成到“即插即用”的图像恢复框架中，由此产生的即插即用流匹配（PnP-Flow）模型在图像恢复方面取得了显著的实际成功。然而，关于PnP-Flow的理论理解落后于其实际表现。在这篇论文中，我们为PnP-Flow推导出了一个连续极限，并得出了一种随机微分方程（SDE）替代模型来表示PnP-Flow。该SDE模型提供了两个特别的见解，以改进用于图像恢复的PnP-Flow：(1) 它使我们可以量化图像恢复中的误差，告知我们如何优化步长调度并调节神经网络参数化向量场的Lipschitz常数，从而减少误差。(2) 它还告诫我们可以通过外推加速现成的PnP-Flow模型，这导致了提议SDE模型的一个重缩放版本。我们使用包括图像去噪、去模糊、超分辨率和插值在内的多个基准任务来验证增强版PnP-Flow（基于SDE）的有效性。数值结果显示，我们的方法在评估指标上显著优于基线的PnP-Flow和其他最先进方法，实现了卓越的整体性能。
URL
https://arxiv.org/abs/2512.04283
PDF
-
Fast & Efficient Normalizing Flows and Applications of Image Generative Models
2025-12-03 18:29:03Sandeep NagarAbstract
This thesis presents novel contributions in two primary areas: advancing the efficiency of generative models, particularly normalizing flows, and applying generative models to solve real-world computer vision challenges. The first part introduce significant improvements to normalizing flow architectures through six key innovations: 1) Development of invertible 3x3 Convolution layers with mathematically proven necessary and sufficient conditions for invertibility, (2) introduction of a more efficient Quad-coupling layer, 3) Design of a fast and efficient parallel inversion algorithm for kxk convolutional layers, 4) Fast & efficient backpropagation algorithm for inverse of convolution, 5) Using inverse of convolution, in Inverse-Flow, for the forward pass and training it using proposed backpropagation algorithm, and 6) Affine-StableSR, a compact and efficient super-resolution model that leverages pre-trained weights and Normalizing Flow layers to reduce parameter count while maintaining performance. The second part: 1) An automated quality assessment system for agricultural produce using Conditional GANs to address class imbalance, data scarcity and annotation challenges, achieving good accuracy in seed purity testing; 2) An unsupervised geological mapping framework utilizing stacked autoencoders for dimensionality reduction, showing improved feature extraction compared to conventional methods; 3) We proposed a privacy preserving method for autonomous driving datasets using on face detection and image inpainting; 4) Utilizing Stable Diffusion based image inpainting for replacing the detected face and license plate to advancing privacy-preserving techniques and ethical considerations in the field.; and 5) An adapted diffusion model for art restoration that effectively handles multiple types of degradation through unified fine-tuning.
Abstract (translated)
这篇论文在两个主要领域提出了创新贡献：一是提升生成模型（尤其是归一化流）的效率，二是应用生成模型解决实际计算机视觉挑战。第一部分通过六项关键创新对归一化流动架构进行了重大改进： 1) 开发了具有数学证明的必要和充分条件以确保可逆性的可逆3x3卷积层。 2) 引入了一种更高效的四耦合层（Quad-coupling layer）。 3) 设计了一个快速且有效的并行反演算法，用于kxk卷积层。 4) 为卷积的逆向传播开发了高效算法。 5) 在前向传递中使用卷积的逆进行Invert-Flow，并通过提出的反向传播算法进行训练。 6) Affine-StableSR，一个紧凑高效的超分辨率模型，利用预训练权重和归一化流动层来减少参数数量同时保持性能。 第二部分包括： 1) 一种使用条件生成对抗网络（Conditional GANs）的自动质量评估系统，用于解决农业产品中的类别不平衡、数据稀缺和标注挑战，并在种子纯度测试中实现了良好的准确性。 2) 一个利用堆叠自编码器进行降维的无监督地质制图框架，在特征提取方面优于传统方法。 3) 提出了一种使用面部检测和图像修复来保护自动驾驶数据集隐私的方法。 4) 利用基于Stable Diffusion的图像修复技术，将检测到的脸部和车牌替换为提高隐私保护技术和伦理考虑的方法。 5) 一种经过改进的扩散模型用于艺术作品恢复，能够通过统一微调处理多种类型的退化。
URL
https://arxiv.org/abs/2512.04039
PDF
-
Beyond the Ground Truth: Enhanced Supervision for Image Restoration
2025-12-03 16:30:32Donghun Ryou, Inju Ha, Sanghyeok Chu, Bohyung HanAbstract
Deep learning-based image restoration has achieved significant success. However, when addressing real-world degradations, model performance is limited by the quality of ground-truth images in datasets due to practical constraints in data acquisition. To address this limitation, we propose a novel framework that enhances existing ground truth images to provide higher-quality supervision for real-world restoration. Our framework generates perceptually enhanced ground truth images using super-resolution by incorporating adaptive frequency masks, which are learned by a conditional frequency mask generator. These masks guide the optimal fusion of frequency components from the original ground truth and its super-resolved variants, yielding enhanced ground truth images. This frequency-domain mixup preserves the semantic consistency of the original content while selectively enriching perceptual details, preventing hallucinated artifacts that could compromise fidelity. The enhanced ground truth images are used to train a lightweight output refinement network that can be seamlessly integrated with existing restoration models. Extensive experiments demonstrate that our approach consistently improves the quality of restored images. We further validate the effectiveness of both supervision enhancement and output refinement through user studies. Code is available at this https URL.
Abstract (translated)
基于深度学习的图像恢复技术已经取得了显著的成功。然而，在处理现实世界中的退化问题时，由于数据采集的实际限制，模型性能受到训练集中真实地面图像质量的影响而受限。为了解决这一局限性，我们提出了一种新颖框架，通过增强现有的地面真相图像来提供更高质量的真实世界图像恢复指导。 我们的框架使用超分辨率技术生成感知上增强的地面真相图像，并结合自适应频率掩码，这些掩码由条件频率掩码生成器学习得到。这些掩码引导原始地面真相和其超级解析变体之间的频域成分的最佳融合，从而产生增强的地面真相图像。这种频域混合保持了原始内容的语义一致性，同时选择性地丰富感知细节，防止可能损害真实性的幻觉伪影。 我们利用生成的增强地面真相图像来训练一个轻量级的输出精炼网络，并可以无缝集成到现有的恢复模型中。大量的实验表明，我们的方法能够持续提高恢复图像的质量。通过用户研究进一步验证了监督提升和输出精炼的有效性。代码可在提供的链接处获取。
URL
https://arxiv.org/abs/2512.03932
PDF
-
EFDiT: Efficient Fine-grained Image Generation Using Diffusion Transformer Models
2025-12-03 14:10:06Kun Wang, Donglin Di, Tonghua Su, Lei FanAbstract
Diffusion models are highly regarded for their controllability and the diversity of images they generate. However, class-conditional generation methods based on diffusion models often focus on more common categories. In large-scale fine-grained image generation, issues of semantic information entanglement and insufficient detail in the generated images still persist. This paper attempts to introduce a concept of a tiered embedder in fine-grained image generation, which integrates semantic information from both super and child classes, allowing the diffusion model to better incorporate semantic information and address the issue of semantic entanglement. To address the issue of insufficient detail in fine-grained images, we introduce the concept of super-resolution during the perceptual information generation stage, enhancing the detailed features of fine-grained images through enhancement and degradation models. Furthermore, we propose an efficient ProAttention mechanism that can be effectively implemented in the diffusion model. We evaluate our method through extensive experiments on public benchmarks, demonstrating that our approach outperforms other state-of-the-art fine-tuning methods in terms of performance.
Abstract (translated)
扩散模型因其可控性和生成图像的多样性而备受推崇。然而，基于扩散模型的条件类别生成方法通常更关注常见类别，而在大规模细粒度图像生成中，语义信息纠缠和生成图像细节不足的问题仍然存在。本文尝试在细粒度图像生成中引入分层嵌入器的概念，该嵌入器结合了超类和子类的语义信息，使扩散模型能够更好地融合语义信息并解决语义纠缠问题。为了解决细粒度图像中细节不足的问题，我们在感知信息生成阶段引入了超分辨率概念，通过增强和退化模型来提升细粒度图像中的详细特征。此外，我们还提出了一种高效的ProAttention机制，该机制可以在扩散模型中有效实现。我们通过对公共基准进行广泛的实验评估，证明我们的方法在性能上优于其他最先进的微调方法。
URL
https://arxiv.org/abs/2512.05152
PDF
