Super_Resolution
-
Multiscale Corrections by Continuous Super-Resolution
2025-12-23 14:44:20Zhi-Song Liu, Roland Maier, Andreas RuppAbstract
Finite element methods typically require a high resolution to satisfactorily approximate micro and even macro patterns of an underlying physical model. This issue can be circumvented by appropriate multiscale strategies that are able to obtain reasonable approximations on under-resolved scales. In this paper, we study the implicit neural representation and propose a continuous super-resolution network as a correction strategy for multiscale effects. It can take coarse finite element data to learn both in-distribution and out-of-distribution high-resolution finite element predictions. Our highlight is the design of a local implicit transformer, which is able to learn multiscale features. We also propose Gabor wavelet-based coordinate encodings, which can overcome the bias of neural networks learning low-frequency features. Finally, perception is often preferred over distortion, so scientists can recognize the visual pattern for further investigation. However, implicit neural representation is known for its lack of local pattern supervision. We propose to use stochastic cosine similarities to compare the local feature differences between prediction and ground truth. It shows better performance on structural alignments. Our experiments show that our proposed strategy achieves superior performance as an in-distribution and out-of-distribution super-resolution strategy.
Abstract (translated)
有限元方法通常需要高分辨率才能满意地近似底层物理模型的微尺度和宏尺度模式。这一问题可以通过适当的多尺度策略来解决，这些策略能够获得在低分辨率下合理近似的解。本文研究了隐式神经表示，并提出了一种连续超分辨率网络作为多重尺度效应的校正策略。该方法可以利用粗糙的有限元数据学习分布内及分布外的高分辨率有限元预测。 我们的亮点在于设计了一个局部隐式变压器，它可以学习多尺度特征。此外，我们还提出了基于Gabor小波的坐标编码，这可以克服神经网络在学习低频特征时产生的偏差。然而，感知通常比失真更受科学家们的青睐，因此他们可以通过视觉模式进一步进行研究。 尽管隐式神经表示缺乏局部模式监督，我们提出使用随机余弦相似度来比较预测和真实值之间的局部特征差异。这种方法在结构对齐上表现更好。 我们的实验表明，在分布内及分布外的超分辨率策略中，所提出的策略表现出优越性。
URL
https://arxiv.org/abs/2411.07576
PDF
-
Field-Space Attention for Structure-Preserving Earth System Transformers
2025-12-23 13:31:21Maximilian Witte, Johannes Meuer, \'Etienne Pl\'esiat, Christopher KadowAbstract
Accurate and physically consistent modeling of Earth system dynamics requires machine-learning architectures that operate directly on continuous geophysical fields and preserve their underlying geometric structure. Here we introduce Field-Space attention, a mechanism for Earth system Transformers that computes attention in the physical domain rather than in a learned latent space. By maintaining all intermediate representations as continuous fields on the sphere, the architecture enables interpretable internal states and facilitates the enforcement of scientific constraints. The model employs a fixed, non-learned multiscale decomposition and learns structure-preserving deformations of the input field, allowing coherent integration of coarse and fine-scale information while avoiding the optimization instabilities characteristic of standard single-scale Vision Transformers. Applied to global temperature super-resolution on a HEALPix grid, Field-Space Transformers converge more rapidly and stably than conventional Vision Transformers and U-Net baselines, while requiring substantially fewer parameters. The explicit preservation of field structure throughout the network allows physical and statistical priors to be embedded directly into the architecture, yielding improved fidelity and reliability in data-driven Earth system modeling. These results position Field-Space Attention as a compact, interpretable, and physically grounded building block for next-generation Earth system prediction and generative modeling frameworks.
Abstract (translated)
地球系统动态的准确和物理一致性的建模需要能够在连续地物理场直接操作并保持其内在几何结构的机器学习架构。在这里，我们介绍了Field-Space注意力机制，这是为地球系统的Transformer设计的一种方法，在该机制中计算是在物理域而非在学习到的潜在空间进行的。通过在整个过程中将所有中间表示维护为球面上的连续字段，这种架构能够支持可解释的内部状态，并促进了科学约束条件的应用。该模型采用了一个固定的、非学习的多尺度分解，并学会了输入场保持结构的变形，从而可以连贯地集成粗粒度和细粒度信息，同时避免了标准单尺度Vision Transformer中的优化不稳定性问题。 当我们应用这种Field-Space Transformer方法对HEALPix网格上的全球温度超分辨率进行处理时，与传统的Vision Transformers以及U-Net基准模型相比，它能够更快、更稳定地收敛，并且所需的参数要少得多。在整个网络中显式保持字段结构允许物理和统计先验被直接嵌入到架构中，从而提高了数据驱动的地球系统建模的准确性和可靠性。 这些结果将Field-Space Attention定位为下一代地球系统预测和生成模型框架中的一个紧凑、可解释且具有物理学基础的重要构建模块。
URL
https://arxiv.org/abs/2512.20350
PDF
-
Super-Resolution Enhancement of Medical Images Based on Diffusion Model: An Optimization Scheme for Low-Resolution Gastric Images
2025-12-22 03:37:47Haozhe JiaAbstract
Capsule endoscopy has enabled minimally invasive gastrointestinal imaging, but its clinical utility is limited by the inherently low resolution of captured images due to hardware, power, and transmission constraints. This limitation hampers the identification of fine-grained mucosal textures and subtle pathological features essential for early diagnosis. This work investigates a diffusion-based super-resolution framework to enhance capsule endoscopy images in a data-driven and anatomically consistent manner. We adopt the SR3 (Super-Resolution via Repeated Refinement) framework built upon Denoising Diffusion Probabilistic Models (DDPMs) to learn a probabilistic mapping from low-resolution to high-resolution images. Unlike GAN-based approaches that often suffer from training instability and hallucination artifacts, diffusion models provide stable likelihood-based training and improved structural fidelity. The HyperKvasir dataset, a large-scale publicly available gastrointestinal endoscopy dataset, is used for training and evaluation. Quantitative results demonstrate that the proposed method significantly outperforms bicubic interpolation and GAN-based super-resolution methods such as ESRGAN, achieving PSNR of 27.5 dB and SSIM of 0.65 for a baseline model, and improving to 29.3 dB and 0.71 with architectural enhancements including attention mechanisms. Qualitative results show improved preservation of anatomical boundaries, vascular patterns, and lesion structures. These findings indicate that diffusion-based super-resolution is a promising approach for enhancing non-invasive medical imaging, particularly in capsule endoscopy where image resolution is fundamentally constrained.
Abstract (translated)
胶囊内镜技术实现了微创胃肠成像，但其临床应用受限于图像固有的低分辨率，这主要是由于硬件、电源和传输的限制。这种局限性阻碍了对细微粘膜纹理及早期诊断所需的小病变特征的识别。本研究探讨了一种基于扩散的超分辨率框架，以数据驱动且解剖学一致的方式增强胶囊内镜图像。我们采用了SR3（通过重复细化实现的超分辨率）框架，并在其基础上构建了去噪扩散概率模型（DDPMs），以此来学习从低分辨率到高分辨率图像的概率映射。与常因训练不稳定和幻觉伪影问题而受困的GAN（生成对抗网络）方法不同，扩散模型提供了基于似然性的稳定训练以及结构保真度的提升。 为了训练和评估，我们使用了HyperKvasir数据集——一个大规模公开可用的胃肠内镜图像数据集。定量结果显示所提出的方法在基准模型上达到了27.5 dB的PSNR（峰值信噪比）和0.65的SSIM（结构相似性指数），显著优于双三次插值及GAN超分辨率方法如ESRGAN；通过加入注意力机制等架构改进，性能进一步提升至29.3 dB PSNR和0.71 SSIM。定性结果表明该模型能更好地保留解剖边界、血管模式以及病变结构。 这些发现表明基于扩散的超分辨率技术是增强非侵入性医学成像的一个有前景的方法，特别是在图像分辨率固有限制的胶囊内镜检查中尤为显著。
URL
https://arxiv.org/abs/2512.22209
PDF
-
Complex Swin Transformer for Accelerating Enhanced SMWI Reconstruction
2025-12-21 08:58:04Muhammad Usman, Sung-Min GhoAbstract
Susceptibility Map Weighted Imaging (SMWI) is an advanced magnetic resonance imaging technique used to detect nigral hyperintensity in Parkinsons disease. However, full resolution SMWI acquisition is limited by long scan times. Efficient reconstruction methods are therefore required to generate high quality SMWI from reduced k space data while preserving diagnostic relevance. In this work, we propose a complex valued Swin Transformer based network for super resolution reconstruction of multi echo MRI data. The proposed method reconstructs high quality SMWI images from low resolution k space inputs. Experimental results demonstrate that the method achieves a structural similarity index of 0.9116 and a mean squared error of 0.076 when reconstructing SMWI from 256 by 256 k space data, while maintaining critical diagnostic features. This approach enables high quality SMWI reconstruction from reduced k space sampling, leading to shorter scan times without compromising diagnostic detail. The proposed method has the potential to improve the clinical applicability of SMWI for Parkinsons disease and support faster and more efficient neuroimaging workflows.
Abstract (translated)
Susceptibility Map Weighted Imaging (SMWI) 是一种先进的磁共振成像技术，用于检测帕金森病中的黑质高信号。然而，全分辨率的 SMWI 获取受到长时间扫描的影响。因此，需要高效的重建方法来利用减少的空间(k空间)数据生成高质量的 SMWI 图像并保持诊断相关性。 在这项工作中，我们提出了一种基于复数Swin Transformer网络的方法，用于多回波MRI数据的超分辨率重建。该方法可以从低分辨率k空间输入中重构出高质量的SMWI图像。实验结果表明，在从256×256 k空间数据重构 SMWI 时，所提方法在结构相似性指数 (SSIM) 和均方误差 (MSE) 分别达到0.9116和0.076的同时，保持了关键的诊断特征。这种方法使SMWI可以从减少的空间采样中进行高质量重建，从而缩短扫描时间而不损害诊断细节。 该方法有望提高 SMWI 在帕金森病临床应用中的适用性，并支持更快、更高效的神经影像工作流程。
URL
https://arxiv.org/abs/2512.22202
PDF
-
An Empirical Study of Sampling Hyperparameters in Diffusion-Based Super-Resolution
2025-12-19 15:17:12Yudhistira Arief WibowoAbstract
Diffusion models have shown strong potential for solving inverse problems such as single-image super-resolution, where a high-resolution image is recovered from a low-resolution observation using a pretrained unconditional prior. Conditioning methods, including Diffusion Posterior Sampling (DPS) and Manifold Constrained Gradient (MCG), can substantially improve reconstruction quality, but they introduce additional hyperparameters that require careful tuning. In this work, we conduct an empirical ablation study on FFHQ super-resolution to identify the dominant factors affecting performance when applying conditioning to pretrained diffusion models, and show that the conditioning step size has a significantly greater impact than the diffusion step count, with step sizes in the range of [2.0, 3.0] yielding the best overall performance in our experiments.
Abstract (translated)
扩散模型在解决诸如单图像超分辨率等逆向问题方面展现出强大的潜力，即通过预训练的无条件先验从低分辨率观测中恢复高分辨率图像。条件方法（包括扩散后验采样(DPS)和流形约束梯度(MCG)）可以显著提高重建质量，但它们引入了需要仔细调整的额外超参数。在这项工作中，我们通过对FFHQ超分辨率进行实证消融研究，识别出在将条件应用于预训练扩散模型时影响性能的主要因素，并表明条件步骤大小比扩散步数对性能的影响更大，在我们的实验中，范围为[2.0, 3.0]的步骤大小产生了最佳的整体效果。
URL
https://arxiv.org/abs/2512.17675
PDF
-
Self-Supervised Weighted Image Guided Quantitative MRI Super-Resolution
2025-12-19 14:15:31Alireza Samadifardheris, Dirk H. J. Poot, Florian Wiesinger, Stefan Klein, Juan A. Hernandez-TamamesAbstract
High-resolution (HR) quantitative MRI (qMRI) relaxometry provides objective tissue characterization but remains clinically underutilized due to lengthy acquisition times. We propose a physics-informed, self-supervised framework for qMRI super-resolution that uses routinely acquired HR weighted MRI (wMRI) scans as guidance, thus, removing the necessity for HR qMRI ground truth during training. We formulate super-resolution as Bayesian maximum a posteriori inference, minimizing two discrepancies: (1) between HR images synthesized from super-resolved qMRI maps and acquired wMRI guides via forward signal models, and (2) between acquired LR qMRI and downsampled predictions. This physics-informed objective allows the models to learn from clinical wMRI without HR qMRI supervision. To validate the concept, we generate training data by synthesizing wMRI guides from HR qMRI using signal equations, then degrading qMRI resolution via k-space truncation. A deep neural network learns the super-resolution mapping. Ablation experiments demonstrate that T1-weighted images primarily enhance T1 maps, T2-weighted images improve T2 maps, and combined guidance optimally enhances all parameters simultaneously. Validation on independently acquired in-vivo data from a different qMRI sequence confirms cross-qMRI sequence generalizability. Models trained on synthetic data can produce super-resolved maps from a 1-minute acquisition with quality comparable to a 5-minute reference scan, leveraging the scanner-independent nature of relaxometry parameters. By decoupling training from HR qMRI requirement, our framework enables fast qMRI acquisitions enhanced via routine clinical images, offering a practical pathway for integrating quantitative relaxometry into clinical workflows with acceptable additional scan time.
Abstract (translated)
高分辨率（HR）定量磁共振成像（qMRI）弛豫测量能够提供客观的组织特征描述，但由于采集时间过长，在临床上的应用相对较少。我们提出了一种基于物理信息的自监督框架，用于qMRI超分辨率处理，该框架使用常规获取的HR加权MRI (wMRI)扫描作为指导，因此在训练过程中不需要高分辨率qMRI的真实数据。我们将超分辨率问题形式化为贝叶斯最大后验概率推理，并最小化两种差异：（1）从超分辨率qMRI映射合成的HR图像与通过前向信号模型获得的wMRI引导之间的差异；（2）实际低分辨率（LR）qMRI和下采样预测之间的差异。这种基于物理信息的目标使得模型能够在没有高分辨率qMRI监督的情况下，从临床wMRI中学习。 为了验证这一概念的有效性，我们生成训练数据的方法是：通过信号方程从HR qMRI合成wMRI引导，然后通过对k空间截断来降低qMRI的分辨率。深度神经网络学习超分辨率映射。消融实验表明，T1加权图像主要增强T1图谱，T2加权图像改善T2图谱，而联合指导则最优地同时增强了所有参数。 在独立获得的不同qMRI序列的体内数据上进行验证，证明了模型具有跨qMRI序列的泛化能力。基于合成数据训练得到的模型可以从一个1分钟采集的数据中生成与5分钟参考扫描质量相当的超分辨率图谱，这得益于弛豫参数不受扫描仪限制的特性。 通过将训练过程从高分辨率qMRI需求中解耦出来，我们的框架能够利用常规临床图像来加速qMRI采集。这一方法提供了一条实用路径，可以在可接受的额外扫描时间内，将定量松弛测量集成到临床工作流程中。
URL
https://arxiv.org/abs/2512.17612
PDF
-
Multi-level distortion-aware deformable network for omnidirectional image super-resolution
2025-12-19 08:35:08Cuixin Yang, Rongkang Dong, Kin-Man Lam, Yuhang Zhang, Guoping QiuAbstract
As augmented reality and virtual reality applications gain popularity, image processing for OmniDirectional Images (ODIs) has attracted increasing attention. OmniDirectional Image Super-Resolution (ODISR) is a promising technique for enhancing the visual quality of ODIs. Before performing super-resolution, ODIs are typically projected from a spherical surface onto a plane using EquiRectangular Projection (ERP). This projection introduces latitude-dependent geometric distortion in ERP images: distortion is minimal near the equator but becomes severe toward the poles, where image content is stretched across a wider area. However, existing ODISR methods have limited sampling ranges and feature extraction capabilities, which hinder their ability to capture distorted patterns over large areas. To address this issue, we propose a novel Multi-level Distortion-aware Deformable Network (MDDN) for ODISR, designed to expand the sampling range and receptive field. Specifically, the feature extractor in MDDN comprises three parallel branches: a deformable attention mechanism (serving as the dilation=1 path) and two dilated deformable convolutions with dilation rates of 2 and 3. This architecture expands the sampling range to include more distorted patterns across wider areas, generating dense and comprehensive features that effectively capture geometric distortions in ERP images. The representations extracted from these deformable feature extractors are adaptively fused in a multi-level feature fusion module. Furthermore, to reduce computational cost, a low-rank decomposition strategy is applied to dilated deformable convolutions. Extensive experiments on publicly available datasets demonstrate that MDDN outperforms state-of-the-art methods, underscoring its effectiveness and superiority in ODISR.
Abstract (translated)
随着增强现实和虚拟现实应用的普及，全景图像（OmniDirectional Images, ODIs）的图像处理技术越来越受到关注。OmniDirectional Image Super-Resolution (ODISR) 是一种有前景的技术，用于提高 ODIs 的视觉质量。在进行超分辨率处理之前，通常会将 ODIs 从球面投影到平面上，采用的是等矩形投影（EquiRectangular Projection, ERP）。这种投影会在 ERP 图像中引入纬度相关的几何失真：在赤道附近失真较小，而在两极地区则显著增加，导致图像内容在更广泛的区域内拉伸。然而，现有的 ODISR 方法具有有限的采样范围和特征提取能力，这限制了它们捕捉大范围内失真模式的能力。为了解决这一问题，我们提出了一种新颖的多级扭曲感知变形网络（Multi-level Distortion-aware Deformable Network, MDDN），旨在扩大采样范围和感受野。具体而言，MDDN 中的特征提取器包括三个并行分支：一个可变形注意力机制（作为 dilation=1 的路径）以及两个具有扩张率为 2 和 3 的可变形膨胀卷积。这种架构扩展了采样范围，能够覆盖更广泛的失真模式，生成密集且全面的特征，有效捕捉 ERP 图像中的几何扭曲。从这些可变形特征提取器中抽取的表示在多级特征融合模块中自适应地进行融合。此外，为了减少计算成本，在扩张可变形卷积上应用了低秩分解策略。在公共数据集上的广泛实验表明，MDDN 在 ODISR 方面优于最先进的方法，证明了其有效性和优越性。
URL
https://arxiv.org/abs/2512.17343
PDF
-
CogSR: Semantic-Aware Speech Super-Resolution via Chain-of-Thought Guided Flow Matching
2025-12-18 08:46:21Jiajun Yuan, Xiaochen Wang, Yuhang Xiao, Yulin Wu, Chenhao Hu, Xueyang LvAbstract
Applying speech super-resolution (SR) to recordings with severely low sampling rates is a critical challenge in digital archiving and investigative audio recovery. In these scenarios, the input lacks essential acoustic cues. Consequently, existing generative models often fail; without sufficient context, they hallucinate phonetic content, guessing words based on probability rather than meaning. To address this, we propose CogSR, a framework designed specifically for high-precision, offline restoration. Our approach shifts the focus from simple signal mapping to cognitive reconstruction. By integrating a Large Audio-Language Model, we employ Chain-of-Thought reasoning to act as a semantic anchor, while explicit acoustic priors ensure the speaker's identity remains consistent. This guides a Rectified Flow backbone to synthesize high-frequency details that are not only realistic but linguistically accurate. Evaluations show that CogSR effectively eliminates ambiguity in severe degradation regimes, making it a robust solution for restoring high-value legacy and surveillance audio.
Abstract (translated)
将语音超分辨率（SR）技术应用于采样率极低的录音，在数字档案和调查音频恢复领域是一个关键挑战。在这种情况下，输入缺乏重要的声学线索。因此，现有的生成模型往往失败；由于缺少足够的上下文信息，它们会臆造出可能不符合实际意义的声音内容。为了解决这一问题，我们提出了一种名为CogSR的框架，该框架专门用于高精度、离线恢复任务。我们的方法将重点从简单的信号映射转移到认知重建上。通过整合大型音频-语言模型，我们利用链式思维推理充当语义锚点，同时显式的声学先验确保说话人的身份一致性。这引导了一个校正流主干网络合成高频细节，这些细节不仅现实而且在语音学上准确无误。 评估结果显示，CogSR能够有效消除严重退化情况下的模糊性，使其成为恢复高价值的历史和监控音频的可靠解决方案。
URL
https://arxiv.org/abs/2512.16304
PDF
-
Pixel Super-Resolved Fluorescence Lifetime Imaging Using Deep Learning
2025-12-18 07:28:10Paloma Casteleiro Costa, Parnian Ghapandar Kashani, Xuhui Liu, Alexander Chen, Ary Portes, Julien Bec, Laura Marcu, Aydogan OzcanAbstract
Fluorescence lifetime imaging microscopy (FLIM) is a powerful quantitative technique that provides metabolic and molecular contrast, offering strong translational potential for label-free, real-time diagnostics. However, its clinical adoption remains limited by long pixel dwell times and low signal-to-noise ratio (SNR), which impose a stricter resolution-speed trade-off than conventional optical imaging approaches. Here, we introduce FLIM_PSR_k, a deep learning-based multi-channel pixel super-resolution (PSR) framework that reconstructs high-resolution FLIM images from data acquired with up to a 5-fold increased pixel size. The model is trained using the conditional generative adversarial network (cGAN) framework, which, compared to diffusion model-based alternatives, delivers a more robust PSR reconstruction with substantially shorter inference times, a crucial advantage for practical deployment. FLIM_PSR_k not only enables faster image acquisition but can also alleviate SNR limitations in autofluorescence-based FLIM. Blind testing on held-out patient-derived tumor tissue samples demonstrates that FLIM_PSR_k reliably achieves a super-resolution factor of k = 5, resulting in a 25-fold increase in the space-bandwidth product of the output images and revealing fine architectural features lost in lower-resolution inputs, with statistically significant improvements across various image quality metrics. By increasing FLIM's effective spatial resolution, FLIM_PSR_k advances lifetime imaging toward faster, higher-resolution, and hardware-flexible implementations compatible with low-numerical-aperture and miniaturized platforms, better positioning FLIM for translational applications.
Abstract (translated)
荧光寿命成像显微镜（FLIM）是一种强大的定量技术，可提供代谢和分子对比度，并且具有重要的转化潜力，可用于无标签实时诊断。然而，由于像素停留时间长以及信噪比低，其临床应用受到了限制，这些因素对分辨率与速度的权衡要求比传统光学成像方法更为严格。在这里，我们引入了FLIM_PSR_k，这是一种基于深度学习的多通道像素超分辨率（PSR）框架，可以从使用高达5倍像素尺寸采集的数据中重建高分辨率的FLIM图像。该模型采用条件生成对抗网络（cGAN）架构进行训练，相比于基于扩散模型的方法，在提供更稳健的PSR重建的同时，还具有显著缩短推理时间的优势，这对于实际部署至关重要。 FLIM_PSR_k不仅加快了成像速度，还可以缓解自体荧光基FLIM中的信噪比限制。在未公开的患者来源肿瘤组织样本上的盲测结果表明，FLIM_PSR_k可靠地实现了k=5倍超分辨率，从而将输出图像的空间带宽产品提高了25倍，并揭示了低分辨率输入中丢失的精细结构特征，在各种图像质量指标上均有统计学意义上的显著改善。通过提高FLIM的有效空间分辨率，FLIM_PSR_k推进了寿命成像技术朝向更快、更高分辨率以及硬件灵活兼容的方向发展，尤其适用于低数值孔径和微型化平台，并且更好地将FLIM定位为转化应用的理想选择。
URL
https://arxiv.org/abs/2512.16266
PDF
-
MMMamba: A Versatile Cross-Modal In Context Fusion Framework for Pan-Sharpening and Zero-Shot Image Enhancement
2025-12-17 10:07:09Yingying Wang, Xuanhua He, Chen Wu, Jialing Huang, Suiyun Zhang, Rui Liu, Xinghao Ding, Haoxuan CheAbstract
Pan-sharpening aims to generate high-resolution multispectral (HRMS) images by integrating a high-resolution panchromatic (PAN) image with its corresponding low-resolution multispectral (MS) image. To achieve effective fusion, it is crucial to fully exploit the complementary information between the two modalities. Traditional CNN-based methods typically rely on channel-wise concatenation with fixed convolutional operators, which limits their adaptability to diverse spatial and spectral variations. While cross-attention mechanisms enable global interactions, they are computationally inefficient and may dilute fine-grained correspondences, making it difficult to capture complex semantic relationships. Recent advances in the Multimodal Diffusion Transformer (MMDiT) architecture have demonstrated impressive success in image generation and editing tasks. Unlike cross-attention, MMDiT employs in-context conditioning to facilitate more direct and efficient cross-modal information exchange. In this paper, we propose MMMamba, a cross-modal in-context fusion framework for pan-sharpening, with the flexibility to support image super-resolution in a zero-shot manner. Built upon the Mamba architecture, our design ensures linear computational complexity while maintaining strong cross-modal interaction capacity. Furthermore, we introduce a novel multimodal interleaved (MI) scanning mechanism that facilitates effective information exchange between the PAN and MS modalities. Extensive experiments demonstrate the superior performance of our method compared to existing state-of-the-art (SOTA) techniques across multiple tasks and benchmarks.
Abstract (translated)
光谱锐化（Pan-sharpening）的目标是通过整合高分辨率全色（PAN）图像和对应的低分辨率多光谱（MS）图像，生成高质量的高分辨多光谱（HRMS）图像。为了实现有效的融合，充分挖掘这两种模态之间的互补信息至关重要。传统的基于CNN的方法通常依赖于通道级串联以及固定的卷积操作符，这限制了它们适应各种空间和光谱变化的能力。虽然交叉注意力机制能够提供全局交互作用，但其计算效率较低，并且可能稀释精细对应关系，难以捕捉复杂的语义关系。 最近，在多模态扩散变压器（MMDiT）架构上的进展在图像生成和编辑任务中取得了令人印象深刻的成功表现。不同于交叉注意力机制，MMDiT采用上下文条件化以促进更加直接高效的跨模态信息交换。本文提出了一种新的光谱锐化框架——MMMamba，这是一种基于Mamba架构的跨模态上下文融合框架，并且能够支持零样本图像超分辨率任务。 我们的设计在确保线性计算复杂度的同时，仍然保持了强大的跨模态交互能力。此外，我们还引入了一个新颖的多模态交错（MI）扫描机制，以促进PAN和MS模态之间的有效信息交换。广泛的实验结果证明了我们的方法在多个任务和基准上的性能优于现有的最先进（SOTA）技术。
URL
https://arxiv.org/abs/2512.15261
PDF
-
MVGSR: Multi-View Consistent 3D Gaussian Super-Resolution via Epipolar Guidance
2025-12-17 03:23:12Kaizhe Zhang, Shinan Chen, Qian Zhao, Weizhan Zhang, Caixia Yan, Yudeng XinAbstract
Scenes reconstructed by 3D Gaussian Splatting (3DGS) trained on low-resolution (LR) images are unsuitable for high-resolution (HR) rendering. Consequently, a 3DGS super-resolution (SR) method is needed to bridge LR inputs and HR rendering. Early 3DGS SR methods rely on single-image SR networks, which lack cross-view consistency and fail to fuse complementary information across views. More recent video-based SR approaches attempt to address this limitation but require strictly sequential frames, limiting their applicability to unstructured multi-view datasets. In this work, we introduce Multi-View Consistent 3D Gaussian Splatting Super-Resolution (MVGSR), a framework that focuses on integrating multi-view information for 3DGS rendering with high-frequency details and enhanced consistency. We first propose an Auxiliary View Selection Method based on camera poses, making our method adaptable for arbitrarily organized multi-view datasets without the need of temporal continuity or data reordering. Furthermore, we introduce, for the first time, an epipolar-constrained multi-view attention mechanism into 3DGS SR, which serves as the core of our proposed multi-view SR network. This design enables the model to selectively aggregate consistent information from auxiliary views, enhancing the geometric consistency and detail fidelity of 3DGS representations. Extensive experiments demonstrate that our method achieves state-of-the-art performance on both object-centric and scene-level 3DGS SR benchmarks.
Abstract (translated)
使用低分辨率（LR）图像训练的3D高斯点云重建方法（3D Gaussian Splatting，简称3DGS）所重构的场景对于高清渲染来说是不合适的。因此需要一种3DGS超分辨技术来连接低分辨率输入和高质量的高清渲染。早期的3DGS超分辨技术主要依赖于单图像超分辨网络，这些技术缺乏视图间的一致性，并且无法融合来自不同视角的互补信息。最近一些基于视频序列的技术试图解决这一问题，但要求严格的顺序帧输入，限制了其在非结构化多视图数据集中的应用。 本文中我们引入了一种新的方法：多视角一致3D高斯点云超分辨技术（MVGSR），该框架旨在整合多视角信息以提高基于3DGS渲染的高频细节和一致性。首先，我们提出一种基于摄像机姿态的辅助视图选择方法，使我们的方法能够适应任意组织化的多视角数据集，并且无需时间连续性或数据重新排序。 此外，我们首次在3DGS超分辨技术中引入了符合基线约束的多视角注意力机制，这是我们的多视角超分辨网络的核心设计。这种设计使得模型可以有选择地从辅助视图中聚合一致的信息，从而增强了3DGS表示的几何一致性与细节准确性。 大量的实验表明，在面向对象和场景级别的基于3DGS的超分辨率基准测试上，该方法达到了最先进的性能表现。
URL
https://arxiv.org/abs/2512.15048
PDF
-
TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration
2025-12-16 16:25:47Zhiwen Yang, Jiaju Zhang, Yang Yi, Jian Liang, Bingzheng Wei, Yan XuAbstract
Medical image restoration (MedIR) aims to recover high-quality medical images from their low-quality counterparts. Recent advancements in MedIR have focused on All-in-One models capable of simultaneously addressing multiple different MedIR tasks. However, due to significant differences in both modality and degradation types, using a shared model for these diverse tasks requires careful consideration of two critical inter-task relationships: task interference, which occurs when conflicting gradient update directions arise across tasks on the same parameter, and task imbalance, which refers to uneven optimization caused by varying learning difficulties inherent to each task. To address these challenges, we propose a task-adaptive Transformer (TAT), a novel framework that dynamically adapts to different tasks through two key innovations. First, a task-adaptive weight generation strategy is introduced to mitigate task interference by generating task-specific weight parameters for each task, thereby eliminating potential gradient conflicts on shared weight parameters. Second, a task-adaptive loss balancing strategy is introduced to dynamically adjust loss weights based on task-specific learning difficulties, preventing task domination or undertraining. Extensive experiments demonstrate that our proposed TAT achieves state-of-the-art performance in three MedIR tasks--PET synthesis, CT denoising, and MRI super-resolution--both in task-specific and All-in-One settings. Code is available at this https URL.
Abstract (translated)
医学图像恢复（MedIR）的目标是从低质量的医学图像中恢复出高质量的图像。最近，MedIR领域的发展集中在能够同时处理多种不同任务的全功能模型上。然而，由于不同的成像模态和退化类型之间存在显著差异，使用同一共享模型来应对这些多样化任务需要仔细考虑两个关键的任务间关系：即任务干扰（当不同任务在同一参数上的梯度更新方向发生冲突时产生）以及任务不平衡（由各个任务固有的学习难度不均造成的优化不平衡）。为了解决这些问题，我们提出了一种任务自适应的变换器（TAT），这是一种通过两种创新方法动态调整以应对不同任务的新框架。首先，引入了任务自适应权重生成策略来减轻任务干扰，该策略针对每个任务生成特定的任务权重参数，从而消除了共享权重参数上的潜在梯度冲突。其次，我们提出了一个任务自适应损失平衡策略，根据各个任务的具体学习难度动态调整损失权重，防止任务的主导或欠训练现象。 通过广泛的实验表明，在PET合成、CT去噪和MRI超分辨率这三种MedIR任务中，我们的TAT框架在特定任务设置以及全功能模型设置下均达到了最先进的性能水平。代码可在提供的链接处获取。
URL
https://arxiv.org/abs/2512.14550
PDF
-
Bridging Fidelity-Reality with Controllable One-Step Diffusion for Image Super-Resolution
2025-12-16 03:56:02Hao Chen, Junyang Chen, Jinshan Pan, Jiangxin DongAbstract
Recent diffusion-based one-step methods have shown remarkable progress in the field of image super-resolution, yet they remain constrained by three critical limitations: (1) inferior fidelity performance caused by the information loss from compression encoding of low-quality (LQ) inputs; (2) insufficient region-discriminative activation of generative priors; (3) misalignment between text prompts and their corresponding semantic regions. To address these limitations, we propose CODSR, a controllable one-step diffusion network for image super-resolution. First, we propose an LQ-guided feature modulation module that leverages original uncompressed information from LQ inputs to provide high-fidelity conditioning for the diffusion process. We then develop a region-adaptive generative prior activation method to effectively enhance perceptual richness without sacrificing local structural fidelity. Finally, we employ a text-matching guidance strategy to fully harness the conditioning potential of text prompts. Extensive experiments demonstrate that CODSR achieves superior perceptual quality and competitive fidelity compared with state-of-the-art methods with efficient one-step inference.
Abstract (translated)
最近基于扩散的一步法在图像超分辨率领域取得了显著进步，但仍受三个关键限制的制约：(1) 由于对低质量（LQ）输入进行压缩编码导致的信息丢失而产生的较低保真度性能；(2) 生成功能未能充分激活区域识别功能；(3) 文本提示与其相应语义区域之间的不匹配。为解决这些问题，我们提出了CODSR，一种用于图像超分辨率的可控一步扩散网络。首先，我们提出了一种LQ引导特征调节模块，该模块利用来自LQ输入的原始未压缩信息来为扩散过程提供高保真度条件设置。然后，我们开发了一种区域自适应生成功能激活方法，以有效增强感知丰富性而不牺牲局部结构保真度。最后，我们采用一种文本匹配指导策略，充分利用文本提示的调节潜力。广泛的实验表明，CODSR在视觉质量和保真度方面优于最先进的方法，并且其一步推理效率非常高。
URL
https://arxiv.org/abs/2512.14061
PDF
-
Coarse-to-Fine Hierarchical Alignment for UAV-based Human Detection using Diffusion Models
2025-12-15 19:57:36Wenda Li, Meng Wu, Sungmin Eum, Heesung Kwon, Qing QuAbstract
Training object detectors demands extensive, task-specific annotations, yet this requirement becomes impractical in UAV-based human detection due to constantly shifting target distributions and the scarcity of labeled images. As a remedy, synthetic simulators are adopted to generate annotated data, with a low annotation cost. However, the domain gap between synthetic and real images hinders the model from being effectively applied to the target domain. Accordingly, we introduce Coarse-to-Fine Hierarchical Alignment (CFHA), a three-stage diffusion-based framework designed to transform synthetic data for UAV-based human detection, narrowing the domain gap while preserving the original synthetic labels. CFHA explicitly decouples global style and local content domain discrepancies and bridges those gaps using three modules: (1) Global Style Transfer -- a diffusion model aligns color, illumination, and texture statistics of synthetic images to the realistic style, using only a small real reference set; (2) Local Refinement -- a super-resolution diffusion model is used to facilitate fine-grained and photorealistic details for the small objects, such as human instances, preserving shape and boundary integrity; (3) Hallucination Removal -- a module that filters out human instances whose visual attributes do not align with real-world data to make the human appearance closer to the target distribution. Extensive experiments on public UAV Sim2Real detection benchmarks demonstrate that our methods significantly improve the detection accuracy compared to the non-transformed baselines. Specifically, our method achieves up to $+14.1$ improvement of mAP50 on Semantic-Drone benchmark. Ablation studies confirm the complementary roles of the global and local stages and highlight the importance of hierarchical alignment. The code is released at \href{this https URL}{this url}.
Abstract (translated)
训练目标检测器需要大量的、特定任务的标注数据，但在基于无人机的人类检测中，由于目标分布不断变化以及标记图像稀少，这一要求变得不切实际。作为解决方案，合成模拟器被用来生成有注释的数据，这大大降低了标注成本。然而，合成与真实图像之间的领域差距阻碍了模型在目标领域的有效应用。为此，我们引入了一种名为“粗到细层次对齐”（Coarse-to-Fine Hierarchical Alignment, CFHA）的三阶段扩散框架，旨在将合成数据转换为适用于基于无人机的人类检测，并缩小域间差异，同时保持原始标签信息不变。 CFHA明确地解耦了全局风格与局部内容领域的不一致，并通过三个模块来弥补这些差距：（1）全局样式迁移——一个扩散模型仅使用少量的真实参考集，就能将合成图像的颜色、光照和纹理统计特性对齐到真实样式的水平；（2）局部细化——一种超分辨率扩散模型被用来促进小物体（如人类实例）的细粒度和逼真细节，同时保持形状和边界的完整性；（3）幻觉移除——一个过滤模块将视觉属性与现实世界数据不一致的人类实例排除出去，使人类外观更接近目标分布。 在公共无人机Sim2Real检测基准上的广泛实验表明，我们的方法相比于未转换的基线显著提高了检测精度。具体而言，在Semantic-Drone基准上，我们的方法实现了高达+14.1 mAP50的改进。消融研究表明全局和局部阶段之间的互补作用，并强调了层次对齐的重要性。代码可在[此处](this https URL)获取。
URL
https://arxiv.org/abs/2512.13869
PDF
-
Time-aware UNet and super-resolution deep residual networks for spatial downscaling
2025-12-15 08:19:28Mika Sipil\"a, Sabrina Maggio, Sandra De Iaco, Klaus Nordhausen, Monica Palma, Sara TaskinenAbstract
Satellite data of atmospheric pollutants are often available only at coarse spatial resolution, limiting their applicability in local-scale environmental analysis and decision-making. Spatial downscaling methods aim to transform the coarse satellite data into high-resolution fields. In this work, two widely used deep learning architectures, the super-resolution deep residual network (SRDRN) and the encoder-decoder-based UNet, are considered for spatial downscaling of tropospheric ozone. Both methods are extended with a lightweight temporal module, which encodes observation time using either sinusoidal or radial basis function (RBF) encoding, and fuses the temporal features with the spatial representations in the networks. The proposed time-aware extensions are evaluated against their baseline counterparts in a case study on ozone downscaling over Italy. The results suggest that, while only slightly increasing computational complexity, the temporal modules significantly improve downscaling performance and convergence speed.
Abstract (translated)
卫星大气污染物数据通常仅以较低的空间分辨率提供，这限制了它们在局部环境分析和决策中的应用。空间降尺度方法旨在将低分辨率的卫星数据转换为高分辨率的数据场。在这项工作中，考虑使用两种广泛使用的深度学习架构——超分辨率残差网络（SRDRN）和基于编码器-解码器的UNet，来进行对流层臭氧的空间降尺度处理。这两种方法都扩展了一个轻量级的时间模块，该模块通过正弦或径向基函数（RBF）编码来表示观测时间，并将这些时间特征与网络中的空间表示进行融合。在意大利臭氧降尺度案例研究中评估了所提出的具有时间感知功能的扩展方案及其基准模型的表现。结果表明，尽管计算复杂度仅略有增加，但时间模块显著提高了降尺度性能和收敛速度。
URL
https://arxiv.org/abs/2512.13753
PDF
-
Qonvolution: Towards Learning High-Frequency Signals with Queried Convolution
2025-12-15 00:46:09Abhinav Kumar, Tristan Aumentado-Armstrong, Lazar Valkov, Gopal Sharma, Alex Levinshtein, Radek Grzeszczuk, Suren KumarAbstract
Accurately learning high-frequency signals is a challenge in computer vision and graphics, as neural networks often struggle with these signals due to spectral bias or optimization difficulties. While current techniques like Fourier encodings have made great strides in improving performance, there remains scope for improvement when presented with high-frequency information. This paper introduces Queried-Convolutions (Qonvolutions), a simple yet powerful modification using the neighborhood properties of convolution. Qonvolution convolves a low-frequency signal with queries (such as coordinates) to enhance the learning of intricate high-frequency signals. We empirically demonstrate that Qonvolutions enhance performance across a variety of high-frequency learning tasks crucial to both the computer vision and graphics communities, including 1D regression, 2D super-resolution, 2D image regression, and novel view synthesis (NVS). In particular, by combining Gaussian splatting with Qonvolutions for NVS, we showcase state-of-the-art performance on real-world complex scenes, even outperforming powerful radiance field models on image quality.
Abstract (translated)
准确地学习高频信号是计算机视觉和图形学中的一个挑战，因为神经网络由于频谱偏差或优化难题而难以处理这些信号。尽管当前技术如傅里叶编码在提高性能方面取得了重大进展，但在面对高频信息时仍有改进空间。本文介绍了询问式卷积（Queried-Convolutions，简称Qonvolutions），这是一种简单但强大的修改方法，利用了卷积的邻域属性。通过将低频信号与查询（如坐标）进行卷积，Qonvolution能够增强复杂高频信号的学习能力。 我们实证表明，Qonvolutions在一系列对计算机视觉和图形学领域至关重要的高频学习任务中提高了性能，包括1D回归、2D超分辨率、2D图像回归以及新颖视图合成（NVS）。特别地，通过将高斯点喷射与Qonvolution结合用于NVS，我们在真实世界复杂场景上展示了最先进的表现，甚至在图像质量方面超过了强大的辐射场模型。
URL
https://arxiv.org/abs/2512.12898
PDF
-
Composite Classifier-Free Guidance for Multi-Modal Conditioning in Wind Dynamics Super-Resolution
2025-12-13 22:44:41Jacob Schnell, Aditya Makkar, Gunadi Gani, Aniket Srinivasan Ashok, Darren Lo, Mike Optis, Alexander Wong, Yuhao ChenAbstract
Various weather modelling problems (e.g., weather forecasting, optimizing turbine placements, etc.) require ample access to high-resolution, highly accurate wind data. Acquiring such high-resolution wind data, however, remains a challenging and expensive endeavour. Traditional reconstruction approaches are typically either cost-effective or accurate, but not both. Deep learning methods, including diffusion models, have been proposed to resolve this trade-off by leveraging advances in natural image super-resolution. Wind data, however, is distinct from natural images, and wind super-resolvers often use upwards of 10 input channels, significantly more than the usual 3-channel RGB inputs in natural images. To better leverage a large number of conditioning variables in diffusion models, we present a generalization of classifier-free guidance (CFG) to multiple conditioning inputs. Our novel composite classifier-free guidance (CCFG) can be dropped into any pre-trained diffusion model trained with standard CFG dropout. We demonstrate that CCFG outputs are higher-fidelity than those from CFG on wind super-resolution tasks. We present WindDM, a diffusion model trained for industrial-scale wind dynamics reconstruction and leveraging CCFG. WindDM achieves state-of-the-art reconstruction quality among deep learning models and costs up to $1000\times$ less than classical methods.
Abstract (translated)
各种天气建模问题（如天气预报、优化涡轮机布局等）需要大量高分辨率和高度准确的风数据。然而，获取这种高质量的风数据仍然是一个既具挑战性又昂贵的过程。传统的方法通常要么成本效益高但不够准确，要么非常准确但成本高昂。深度学习方法，包括扩散模型（diffusion models），已被提出以利用自然图像超分辨领域的进步来解决这一问题。然而，风数据与自然图像存在本质区别，风数据的超级分辨率通常需要多达10个输入通道，远超过普通3通道RGB自然图像输入。 为了更好地在扩散模型中使用大量的条件变量，我们提出了一个针对多条件输入的无分类器指导（classifier-free guidance, CFG）方法的一般化版本。我们的新型复合无分类器指导（composite classifier-free guidance, CCFG）可以无缝集成到任何预先训练的标准CFG丢弃策略下的扩散模型中。我们在风数据超分辨率任务上展示了，与标准CFG相比，CCFG生成的结果具有更高的保真度。 我们介绍了WindDM，这是一个专门用于工业规模风动态重建的扩散模型，并采用了我们的复合无分类器指导技术。在深度学习模型中，WindDM实现了目前最先进的重建质量，并且其成本相较于传统方法低至千分之一。
URL
https://arxiv.org/abs/2512.13729
PDF
-
CreativeVR: Diffusion-Prior-Guided Approach for Structure and Motion Restoration in Generative and Real Videos
2025-12-12 22:03:14Tejas Panambur, Ishan Rajendrakumar Dave, Chongjian Ge, Ersin Yumer, Xue BaiAbstract
Modern text-to-video (T2V) diffusion models can synthesize visually compelling clips, yet they remain brittle at fine-scale structure: even state-of-the-art generators often produce distorted faces and hands, warped backgrounds, and temporally inconsistent motion. Such severe structural artifacts also appear in very low-quality real-world videos. Classical video restoration and super-resolution (VR/VSR) methods, in contrast, are tuned for synthetic degradations such as blur and downsampling and tend to stabilize these artifacts rather than repair them, while diffusion-prior restorers are usually trained on photometric noise and offer little control over the trade-off between perceptual quality and fidelity. We introduce CreativeVR, a diffusion-prior-guided video restoration framework for AI-generated (AIGC) and real videos with severe structural and temporal artifacts. Our deep-adapter-based method exposes a single precision knob that controls how strongly the model follows the input, smoothly trading off between precise restoration on standard degradations and stronger structure- and motion-corrective behavior on challenging content. Our key novelty is a temporally coherent degradation module used during training, which applies carefully designed transformations that produce realistic structural failures. To evaluate AIGC-artifact restoration, we propose the AIGC54 benchmark with FIQA, semantic and perceptual metrics, and multi-aspect scoring. CreativeVR achieves state-of-the-art results on videos with severe artifacts and performs competitively on standard video restoration benchmarks, while running at practical throughput (about 13 FPS at 720p on a single 80-GB A100). Project page: this https URL.
Abstract (translated)
现代的文本到视频（T2V）扩散模型能够合成视觉上吸引人的片段，但它们在细粒度结构方面仍然很脆弱：即使是最先进的生成器也常常会产生变形的脸部和手部、扭曲的背景以及时间上不一致的动作。这些严重的结构性缺陷同样出现在质量极低的真实世界视频中。相比之下，传统的视频修复与超分辨率（VR/VSR）方法针对的是模糊和下采样等合成退化现象，并倾向于稳定而不是修正这些缺陷，而基于扩散模型的修复方法通常是在光度噪声上进行训练，对感知质量和保真度之间的权衡控制较少。 我们提出了CreativeVR，这是一种基于扩散先验指导的视频修复框架，适用于存在严重结构性和时间性瑕疵的人工智能生成（AIGC）视频以及真实世界视频。我们的深度适配器方法提供了一个单一的精度旋钮，用来控制模型遵循输入数据的程度，从而在标准退化情况下的精确恢复与更具挑战性内容中的结构及运动修正之间平滑地进行权衡。 我们创新的关键在于训练过程中使用的时间一致性退化模块，该模块应用精心设计的转换来生成现实中的结构性失败。为了评估AIGC瑕疵修复，我们提出了AIGC54基准测试集，并采用了FIQA、语义和感知指标以及多方面评分系统。CreativeVR在具有严重缺陷的视频上实现了最佳结果，在标准视频修复基准上的表现也非常出色，同时运行效率高（在单个80GB A100 GPU上以720p分辨率大约为每秒13帧）。 项目页面：[此链接](this https URL)。
URL
https://arxiv.org/abs/2512.12060
PDF
-
Fast and Explicit: Slice-to-Volume Reconstruction via 3D Gaussian Primitives with Analytic Point Spread Function Modeling
2025-12-12 15:01:30Maik Dannecker, Steven Jia, Nil Stolt-Ans\'o, Nadine Girard, Guillaume Auzias, Fran\c{c}ois Rousseau, Daniel RueckertAbstract
Recovering high-fidelity 3D images from sparse or degraded 2D images is a fundamental challenge in medical imaging, with broad applications ranging from 3D ultrasound reconstruction to MRI super-resolution. In the context of fetal MRI, high-resolution 3D reconstruction of the brain from motion-corrupted low-resolution 2D acquisitions is a prerequisite for accurate neurodevelopmental diagnosis. While implicit neural representations (INRs) have recently established state-of-the-art performance in self-supervised slice-to-volume reconstruction (SVR), they suffer from a critical computational bottleneck: accurately modeling the image acquisition physics requires expensive stochastic Monte Carlo sampling to approximate the point spread function (PSF). In this work, we propose a shift from neural network based implicit representations to Gaussian based explicit representations. By parameterizing the HR 3D image volume as a field of anisotropic Gaussian primitives, we leverage the property of Gaussians being closed under convolution and thus derive a \textit{closed-form analytical solution} for the forward model. This formulation reduces the previously intractable acquisition integral to an exact covariance addition ($\mathbf{\Sigma}_{obs} = \mathbf{\Sigma}_{HR} + \mathbf{\Sigma}_{PSF}$), effectively bypassing the need for compute-intensive stochastic sampling while ensuring exact gradient propagation. We demonstrate that our approach matches the reconstruction quality of self-supervised state-of-the-art SVR frameworks while delivering a 5$\times$--10$\times$ speed-up on neonatal and fetal data. With convergence often reached in under 30 seconds, our framework paves the way towards translation into clinical routine of real-time fetal 3D MRI. Code will be public at {this https URL}.
Abstract (translated)
从稀疏或退化的2D图像中恢复高保真度的3D图像是医学成像中的一个基本挑战，其应用范围广泛，涵盖了从3D超声重建到MRI超分辨率等各个领域。在胎儿MRI的情境下，从运动损坏的低分辨率2D采集数据中进行高分辨率3D脑部重建是准确诊断神经发育状况的前提条件。尽管基于隐式神经表示（INRs）的方法最近在自监督切片到体积重建（SVR）方面取得了最先进的性能，但它们面临一个关键的计算瓶颈：精确建模图像获取物理特性需要昂贵的随机蒙特卡洛采样来近似点扩散函数（PSF）。在这项工作中，我们提出了一种从基于神经网络的隐式表示转向基于高斯的显式表示的方法。通过将HR 3D图体积参数化为各向异性高斯基本单元的集合，我们可以利用高斯卷积闭合性质，并推导出前向模型的“封闭形式解析解”。这种表述将之前难以处理的获取积分简化为精确的协方差加法（$\mathbf{\Sigma}_{obs} = \mathbf{\Sigma}_{HR} + \mathbf{\Sigma}_{PSF}$），从而有效地绕过了计算密集型随机采样的需求，同时确保了准确的梯度传播。我们证明我们的方法在重建质量上与最先进的自监督SVR框架相匹配，并且在新生儿和胎儿数据集上的速度提高了5到10倍。通常情况下，在不到30秒的时间内就能达到收敛，这使得我们的框架朝着将实时胎儿3D MRI技术应用于临床实践迈进了一步。 代码将在[此处](https://this-url-will-be-provided)公开发布。
URL
https://arxiv.org/abs/2512.11624
PDF
-
Super-Resolved Canopy Height Mapping from Sentinel-2 Time Series Using LiDAR HD Reference Data across Metropolitan France
2025-12-12 12:49:16Ekaterina Kalinicheva, Florian Helen, St\'ephane Mermoz, Florian Mouret, Milena PlanellsAbstract
Fine-scale forest monitoring is essential for understanding canopy structure and its dynamics, which are key indicators of carbon stocks, biodiversity, and forest health. Deep learning is particularly effective for this task, as it integrates spectral, temporal, and spatial signals that jointly reflect the canopy structure. To address this need, we introduce THREASURE-Net, a novel end-to-end framework for Tree Height Regression And Super-Resolution. The model is trained on Sentinel-2 time series using reference height metrics derived from LiDAR HD data at multiple spatial resolutions over Metropolitan France to produce annual height maps. We evaluate three model variants, producing tree-height predictions at 2.5 m, 5 m, and 10 m resolution. THREASURE-Net does not rely on any pretrained model nor on reference very high resolution optical imagery to train its super-resolution module; instead, it learns solely from LiDAR-derived height information. Our approach outperforms existing state-of-the-art methods based on Sentinel data and is competitive with methods based on very high resolution imagery. It can be deployed to generate high-precision annual canopy-height maps, achieving mean absolute errors of 2.62 m, 2.72 m, and 2.88 m at 2.5 m, 5 m, and 10 m resolution, respectively. These results highlight the potential of THREASURE-Net for scalable and cost-effective structural monitoring of temperate forests using only freely available satellite data. The source code for THREASURE-Net is available at: this https URL.
Abstract (translated)
精细尺度的森林监测对于理解冠层结构及其动态变化至关重要，这些是衡量碳储量、生物多样性和森林健康状况的关键指标。深度学习在此任务中特别有效，因为它能够整合光谱、时间序列和空间信号，这些信号共同反映了冠层结构的情况。为了满足这一需求，我们引入了THREASURE-Net，这是一种新颖的端到端框架，用于树木高度回归及超分辨率生成。该模型利用Sentinel-2卫星的时间序列数据进行训练，并结合来自法国大都会地区多尺度空间分辨率LiDAR高精度数据所推导出的高度指标来生成年度高度图谱。我们评估了三种模型变体，在2.5米、5米和10米的分辨率下分别产生了树木高度预测。THREASURE-Net不依赖任何预训练模型，也不需要非常高的分辨率光学图像来训练其超分辨率模块；相反，它仅从LiDAR衍生的高度信息中学习。 我们的方法在基于Sentinel数据的方法中超越了现有最先进的技术，并且与基于非常高分辨率影像的方法具有竞争力。它可以部署以生成高精度的年度冠层高度图谱，在2.5米、5米和10米的分辨率下分别实现了平均绝对误差为2.62米、2.72米和2.88米的结果。这些结果突显了THREASURE-Net利用仅限于免费获取的卫星数据就具备对温带森林进行可扩展且成本效益高的结构监测的巨大潜力。 THREASURE-Net的源代码可以在以下链接找到：this https URL。
URL
https://arxiv.org/abs/2512.11524
PDF
