Restoration
-
Revisiting the Perception-Distortion Trade-off with Spatial-Semantic Guided Super-Resolution
2026-03-14 20:41:27Dan Wang, Haiyan Sun, Shan Du, Z. Jane Wang, Zhaochong An, Serge Belongie, Xinrui CuiAbstract
Image super-resolution (SR) aims to reconstruct high resolution images with both high perceptual quality and low distortion, but is fundamentally limited by the perception-distortion trade-off. GAN-based SR methods reduce distortion but still struggle with realistic fine-grained textures, whereas diffusion-based approaches synthesize rich details but often deviate from the input, hallucinating structures and degrading fidelity. This tension raises a key challenge: how to exploit the powerful generative priors of diffusion models without sacrificing fidelity. To address this, we propose SpaSemSR, a spatial-semantic guided diffusion framework with two complementary guidances. First, spatial-grounded textual guidance integrates object-level spatial cues with semantic prompts, aligning textual and visual structures to reduce distortion. Second, semantic-enhanced visual guidance with a multi-encoder design and semantic degradation constraints unifies multimodal semantic priors, improving perceptual realism under severe degradations. These complementary guidances are adaptively fused into the diffusion process via spatial-semantic attention, suppressing distortion and hallucination while retaining the strengths of diffusion models. Extensive experiments on multiple benchmarks show that SpaSemSR achieves a superior perception-distortion balance, producing both realistic and faithful restorations.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14112
PDF
-
Self-Supervised Uncertainty Estimation For Super-Resolution of Satellite Images
2026-03-14 18:41:43Zhe Zheng, Val\'ery Dewil, Pablo AriasAbstract
Super-resolution (SR) of satellite imagery is challenging due to the lack of paired low-/high-resolution data. Recent self-supervised SR methods overcome this limitation by exploiting the temporal redundancy in burst observations, but they lack a mechanism to quantify uncertainty in the reconstruction. In this work, we introduce a novel self-supervised loss that allows to estimate uncertainty in image super-resolution without ever accessing the ground-truth high-resolution data. We adopt a decision-theoretic perspective and show that minimizing the corresponding Bayesian risk yields the posterior mean and variance as optimal estimators. We validate our approach on a synthetic SkySat L1B dataset and demonstrate that it produces calibrated uncertainty estimates comparable to supervised methods. Our work bridges self-supervised restoration with uncertainty quantification, making a practical framework for uncertainty-aware image reconstruction.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14074
PDF
-
What Counts as Real? Speech Restoration and Voice Quality Conversion Pose New Challenges to Deepfake Detection
2026-03-14 17:15:31Shree Harsha Bokkahalli Satish, Harm Lameris, Joakim Gustafson, \'Eva Sz\'ekelyAbstract
Audio anti-spoofing systems are typically formulated as binary classifiers distinguishing bona fide from spoofed speech. This assumption fails under layered generative processing, where benign transformations introduce distributional shifts that are misclassified as spoofing. We show that phonation-modifying voice conversion and speech restoration are treated as out-of-distribution despite preserving speaker authenticity. Using a multi-class setup separating bona fide, converted, spoofed, and converted-spoofed speech, we analyse model behaviour through self-supervised learning (SSL) embeddings and acoustic correlates. The benign transformations induce a drift in the SSL space, compressing bona fide and spoofed speech and reducing classifier separability. Reformulating anti-spoofing as a multi-class problem improves robustness to benign shifts while preserving spoof detection, suggesting binary systems model the distribution of raw speech rather than authenticity itself.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14033
PDF
-
CT-Conditioned Diffusion Prior with Physics-Constrained Sampling for PET Super-Resolution
2026-03-14 11:28:23Liutao Yang, Zi Wang, Peiyuan Jing, Xiaowen Wang, Javier A. Montoya-Zegarra, Kuangyu Shi, Daoqiang Zhang, Guang YangAbstract
PET super-resolution is highly under-constrained because paired multi-resolution scans from the same subject are rarely available, and effective resolution is determined by scanner-specific physics (e.g., PSF, detector geometry, and acquisition settings). This limits supervised end-to-end training and makes purely image-domain generative restoration prone to hallucinated structures when anatomical and physical constraints are weak. We formulate PET super-resolution as posterior inference under heterogeneous system configurations and propose a CT-conditioned diffusion framework with physics-constrained sampling. During training, a conditional diffusion prior is learned from high-quality PET/CT pairs using cross-attention for anatomical guidance, without requiring paired LR--HR PET data. During inference, measurement consistency is enforced through a scanner-aware forward model with explicit PSF effects and gradient-based data-consistency refinement. Under both standard and OOD settings, the proposed method consistently improves experimental metrics and lesion-level clinical relevance indicators over strong baselines, while reducing hallucination artifacts and improving structural fidelity.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13901
PDF
-
V-Bridge: Bridging Video Generative Priors to Versatile Few-shot Image Restoration
2026-03-13 15:39:44Shenghe Zheng, Junpeng Jiang, Wenbo LiAbstract
Large-scale video generative models are trained on vast and diverse visual data, enabling them to internalize rich structural, semantic, and dynamic priors of the visual world. While these models have demonstrated impressive generative capability, their potential as general-purpose visual learners remains largely untapped. In this work, we introduce V-Bridge, a framework that bridges this latent capacity to versatile few-shot image restoration tasks. We reinterpret image restoration not as a static regression problem, but as a progressive generative process, and leverage video models to simulate the gradual refinement from degraded inputs to high-fidelity outputs. Surprisingly, with only 1,000 multi-task training samples (less than 2% of existing restoration methods), pretrained video models can be induced to perform competitive image restoration, achieving multiple tasks with a single model, rivaling specialized architectures designed explicitly for this purpose. Our findings reveal that video generative models implicitly learn powerful and transferable restoration priors that can be activated with only extremely limited data, challenging the traditional boundary between generative modeling and low-level vision, and opening a new design paradigm for foundation models in visual tasks.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13089
PDF
-
Empowering Semantic-Sensitive Underwater Image Enhancement with VLM
2026-03-13 08:17:06Guodong Fan, Shengning Zhou, Genji Yuan, Huiyu Li, Jingchun Zhou, Jinjiang LiAbstract
In recent years, learning-based underwater image enhancement (UIE) techniques have rapidly evolved. However, distribution shifts between high-quality enhanced outputs and natural images can hinder semantic cue extraction for downstream vision tasks, thereby limiting the adaptability of existing enhancement models. To address this challenge, this work proposes a new learning mechanism that leverages Vision-Language Models (VLMs) to empower UIE models with semantic-sensitive capabilities. To be concrete, our strategy first generates textual descriptions of key objects from a degraded image via VLMs. Subsequently, a text-image alignment model remaps these relevant descriptions back onto the image to produce a spatial semantic guidance map. This map then steers the UIE network through a dual-guidance mechanism, which combines cross-attention and an explicit alignment loss. This forces the network to focus its restorative power on semantic-sensitive regions during image reconstruction, rather than pursuing a globally uniform improvement, thereby ensuring the faithful restoration of key object features. Experiments confirm that when our strategy is applied to different UIE baselines, significantly boosts their performance on perceptual quality metrics as well as enhances their performance on detection and segmentation tasks, validating its effectiveness and adaptability.
Abstract (translated)
URL
https://arxiv.org/abs/2603.12773
PDF
-
Music Source Restoration with Ensemble Separation and Targeted Reconstruction
2026-03-13 07:40:42Xinlong Deng, Yu Xia, Jie JiangAbstract
The Inaugural Music Source Restoration (MSR) Challenge targets the recovery of original, unprocessed stems from fully mixed and mastered music. Unlike conventional music source separation, MSR requires reversing complex production processes such as equalization, compression, reverberation, and other real-world degradations. To address MSR, we propose a two-stage system. First, an ensemble of pre-trained separation models produces preliminary source estimates. Then a set of pre-trained BSRNN-based restoration models performs targeted reconstruction to refine these estimates. On the official MSR benchmark, our system surpasses the baselines on all metrics, ranking second among all submissions. The code is available at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2603.16926
PDF
-
DINOLight: Robust Ambient Light Normalization with Self-supervised Visual Prior Integration
2026-03-13 02:25:24Youngjin Oh, Junhyeong Kwon, Nam Ik ChoAbstract
This paper presents a new ambient light normalization framework, DINOLight, that integrates the self-supervised model DINOv2's image understanding capability into the restoration process as a visual prior. Ambient light normalization aims to restore images degraded by non-uniform shadows and lighting caused by multiple light sources and complex scene geometries. We observe that DINOv2 can reliably extract both semantic and geometric information from a degraded image. Based on this observation, we develop a novel framework to utilize DINOv2 features for lighting normalization. First, we propose an adaptive feature fusion module that combines features from different DINOv2 layers using a point-wise softmax mask. Next, the fused features are integrated into our proposed restoration network in both spatial and frequency domains through an auxiliary cross-attention mechanism. Experiments show that DINOLight achieves superior performance on the Ambient6K dataset, and that DINOv2 features are effective for enhancing ambient light normalization. We also apply our method to shadow-removal benchmark datasets, achieving competitive results compared to methods that use mask priors. Codes will be released upon acceptance.
Abstract (translated)
URL
https://arxiv.org/abs/2603.12579
PDF
-
CalliMaster: Mastering Page-level Chinese Calligraphy via Layout-guided Spatial Planning
2026-03-12 22:01:19Tianshuo Xu, Tiantian Hong, Zhifei Chen, Fei Chao, Ying-cong ChenAbstract
Page-level calligraphy synthesis requires balancing glyph precision with layout composition. Existing character models lack spatial context, while page-level methods often compromise brushwork detail. In this paper, we present \textbf{CalliMaster}, a unified framework for controllable generation and editing that resolves this conflict by decoupling spatial planning from content synthesis. Inspired by the human cognitive process of ``planning before writing'', we introduce a coarse-to-fine pipeline \textbf{(Text $\rightarrow$ Layout $\rightarrow$ Image)} to tackle the combinatorial complexity of page-scale synthesis. Operating within a single Multimodal Diffusion Transformer, a spatial planning stage first predicts character bounding boxes to establish the global spatial arrangement. This intermediate layout then serves as a geometric prompt for the content synthesis stage, where the same network utilizes flow-matching to render high-fidelity brushwork. Beyond achieving state-of-the-art generation quality, this disentanglement supports versatile downstream capabilities. By treating the layout as a modifiable constraint, CalliMaster enables controllable semantic re-planning: users can resize or reposition characters while the model automatically harmonizes the surrounding void space and brush momentum. Furthermore, we demonstrate the framework's extensibility to artifact restoration and forensic analysis, providing a comprehensive tool for digital cultural heritage.
Abstract (translated)
URL
https://arxiv.org/abs/2603.12482
PDF
-
Towards Universal Computational Aberration Correction in Photographic Cameras: A Comprehensive Benchmark Analysis
2026-03-12 15:53:02Xiaolong Qian, Qi Jiang, Yao Gao, Lei Sun, Zhonghua Yi, Kailun Yang, Luc Van Gool, Kaiwei WangAbstract
Prevalent Computational Aberration Correction (CAC) methods are typically tailored to specific optical systems, leading to poor generalization and labor-intensive re-training for new lenses. Developing CAC paradigms capable of generalizing across diverse photographic lenses offers a promising solution to these challenges. However, efforts to achieve such cross-lens universality within consumer photography are still in their early stages due to the lack of a comprehensive benchmark that encompasses a sufficiently wide range of optical aberrations. Furthermore, it remains unclear which specific factors influence existing CAC methods and how these factors affect their performance. In this paper, we present comprehensive experiments and evaluations involving 24 image restoration and CAC algorithms, utilizing our newly proposed UniCAC, a large-scale benchmark for photographic cameras constructed via automatic optical design. The Optical Degradation Evaluator (ODE) is introduced as a novel framework to objectively assess the difficulty of CAC tasks, offering credible quantification of optical aberrations and enabling reliable evaluation. Drawing on our comparative analysis, we identify three key factors -- prior utilization, network architecture, and training strategy -- that most significantly influence CAC performance, and further investigate their respective effects. We believe that our benchmark, dataset, and observations contribute foundational insights to related areas and lay the groundwork for future investigations. Benchmarks, codes, and Zemax files will be available at this https URL.
Abstract (translated)
目前流行的计算图像校正（CAC）方法通常是为了特定的光学系统量身定制，导致泛化能力差，并且在使用新镜头时需要重新训练以耗费大量人力。开发能够跨多种摄影镜头进行泛化的CAC范式为解决这些问题提供了有前景的方法。然而，在消费级摄影中实现这种跨镜头通用性的努力仍处于初期阶段，原因是缺乏涵盖广泛光学像差的全面基准测试。 此外，尚不清楚影响现有CAC方法的具体因素以及这些因素如何影响其性能。在本文中，我们通过使用新提出的UniCAC大规模基准进行综合实验和评估，涉及24种图像恢复和CAC算法，该基准是通过自动光学设计构建的摄影相机专用的。我们引入了光学退化评估器（ODE），这是一种新的框架，用于客观地评估CAC任务的难度，并提供可信的光学像差量化以及可靠的评估方法。 根据我们的比较分析，我们确定了影响CAC性能的三个关键因素：先验利用、网络架构和训练策略，并进一步研究它们各自的影响。我们认为我们的基准测试、数据集和观察结果为相关领域提供了基础见解，并为未来的调查奠定了基础。基准测试、代码和Zemax文件可在以下网址获取（假设URL已提供，实际使用时请插入正确的URL）。
URL
https://arxiv.org/abs/2603.12083
PDF
-
Derain-Agent: A Plug-and-Play Agent Framework for Rainy Image Restoration
2026-03-12 12:38:23Zhaocheng Yu, Xiang Chen, Runzhe Li, Zihan Geng, Guanglu Sun, Haipeng Li, Kui JiangAbstract
While deep learning has advanced single-image deraining, existing models suffer from a fundamental limitation: they employ a static inference paradigm that fails to adapt to the complex, coupled degradations (e.g., noise artifacts, blur, and color deviation) of real-world rain. Consequently, restored images often exhibit residual artifacts and inconsistent perceptual quality. In this work, we present Derain-Agent, a plug-and-play refinement framework that transitions deraining from static processing to dynamic, agent-based restoration. Derain-Agent equips a base deraining model with two core capabilities: 1) a Planning Network that intelligently schedules an optimal sequence of restoration tools for each instance, and 2) a Strength Modulation mechanism that applies these tools with spatially adaptive intensity. This design enables precise, region-specific correction of residual errors without the prohibitive cost of iterative search. Our method demonstrates strong generalization, consistently boosting the performance of state-of-the-art deraining models on both synthetic and real-world benchmarks.
Abstract (translated)
虽然深度学习在单张图像去雨方面取得了进展，但现有的模型存在一个根本性的局限：它们使用静态的推理范式，无法适应现实世界中降雨带来的复杂、耦合退化（例如噪声痕迹、模糊和颜色偏差）。因此，恢复后的图像常常表现出残留的艺术品，并且感知质量不一致。在本工作中，我们提出了Derain-Agent，这是一个即插即用的优化框架，将去雨处理从静态处理转变为动态代理修复。Derain-Agent为基本的去雨模型配备了两大核心能力：1）一个规划网络，能够智能地为每个实例安排最优的恢复工具序列；2）一个强度调节机制，以空间自适应的方式应用这些工具。这种设计能够在不增加迭代搜索成本的前提下实现精确、区域特定的错误修正。我们的方法展示了强大的泛化能力，在合成和现实世界基准测试中持续提高了最先进的去雨模型的表现水平。
URL
https://arxiv.org/abs/2603.11866
PDF
-
UCAN: Unified Convolutional Attention Network for Expansive Receptive Fields in Lightweight Super-Resolution
2026-03-12 08:46:19Cao Thien Tan, Phan Thi Thu Trang, Do Nghiem Duc, Ho Ngoc Anh, Hanyang Zhuang, Nguyen Duc DungAbstract
Hybrid CNN-Transformer architectures achieve strong results in image super-resolution, but scaling attention windows or convolution kernels significantly increases computational cost, limiting deployment on resource-constrained devices. We present UCAN, a lightweight network that unifies convolution and attention to expand the effective receptive field efficiently. UCAN combines window-based spatial attention with a Hedgehog Attention mechanism to model both local texture and long-range dependencies, and introduces a distillation-based large-kernel module to preserve high-frequency structure without heavy computation. In addition, we employ cross-layer parameter sharing to further reduce complexity. On Manga109 ($4\times$), UCAN-L achieves 31.63 dB PSNR with only 48.4G MACs, surpassing recent lightweight models. On BSDS100, UCAN attains 27.79 dB, outperforming methods with significantly larger models. Extensive experiments show that UCAN achieves a superior trade-off between accuracy, efficiency, and scalability, making it well-suited for practical high-resolution image restoration.
Abstract (translated)
混合CNN-Transformer架构在图像超分辨率任务中取得了很好的结果，但扩大注意力窗口或卷积核会显著增加计算成本，从而限制了其在资源受限设备上的部署。我们提出了UCAN（Unified Convolution and Attention Network），这是一种轻量级网络，它将卷积和注意力统一起来以高效地扩展有效感受野。 UCAN结合了基于窗口的空间注意机制与刺猬注意力机制，以便同时建模局部纹理和长程依赖关系，并引入了一种基于知识蒸馏的大核模块来在不进行大量计算的情况下保留高频结构。此外，我们还采用了跨层参数共享策略进一步降低复杂度。 在Manga109（4倍超分辨率）数据集上，UCAN-L实现了31.63 dB的PSNR值，仅使用了48.4G MACs的操作量，超过了最近的一些轻量级模型。而在BSDS100数据集上，UCAN达到了27.79 dB的成绩，优于那些拥有显著更大规模模型的方法。 通过广泛的实验显示，UCAN在精度、效率和可扩展性之间实现了更佳的平衡，使其非常适合用于实际高分辨率图像恢复任务中。
URL
https://arxiv.org/abs/2603.11680
PDF
-
SEMamba++: A General Speech Restoration Framework Leveraging Global, Local, and Periodic Spectral Patterns
2026-03-12 08:37:28Yongjoon Lee, Jung-Woo ChoiAbstract
General speech restoration demands techniques that can interpret complex speech structures under various distortions. While State-Space Models like SEMamba have advanced the state-of-the-art in speech denoising, they are not inherently optimized for critical speech characteristics, such as spectral periodicity or multi-resolution frequency analysis. In this work, we introduce an architecture tailored to incorporate speech-specific features as inductive biases. In particular, we propose Frequency GLP, a frequency feature extraction block that effectively and efficiently leverages the properties of frequency bins. Then, we design a multi-resolution parallel time-frequency dual-processing block to capture diverse spectral patterns, and a learnable mapping to further enhance model performance. With all our ideas combined, the proposed SEMamba++ achieves the best performance among multiple baseline models while remaining computationally efficient.
Abstract (translated)
一般性的语音恢复技术需要能够在各种失真条件下解析复杂的语音结构的技术。虽然状态空间模型（如SEMamba）已经在语音降噪方面取得了最先进的成果，但它们并不固有地针对诸如频谱周期性或多分辨率频率分析等关键语音特征进行优化。在这项工作中，我们介绍了一种专门设计来整合特定于语音的特性作为归纳偏差的架构。具体来说，我们提出了频率GLP（Frequency GLP），这是一种频率特征提取模块，能够有效且高效地利用频率带的属性。然后，我们设计了一个多分辨率并行时频双处理块，以捕捉多样化的频谱模式，并引入了一种可学习映射来进一步提升模型性能。通过结合所有这些想法，所提出的SEMamba++在保持计算效率的同时，在多个基线模型中取得了最佳性能。
URL
https://arxiv.org/abs/2603.11669
PDF
-
BackdoorIDS: Zero-shot Backdoor Detection for Pretrained Vision Encoder
2026-03-12 08:32:19Siquan Huang, Yijiang Li, Ningzhi Gao, Xingfu Yan, Leyu ShiAbstract
Self-supervised and multimodal vision encoders learn strong visual representations that are widely adopted in downstream vision tasks and large vision-language models (LVLMs). However, downstream users often rely on third-party pretrained encoders with uncertain provenance, exposing them to backdoor attacks. In this work, we propose BackdoorIDS, a simple yet effective zero-shot, inference-time backdoor samples detection method for pretrained vision encoders. BackdoorIDS is motivated by two observations: Attention Hijacking and Restoration. Under progressive input masking, a backdoored image initially concentrates attention on malicious trigger features. Once the masking ratio exceeds the trigger's robustness threshold, the trigger is deactivated, and attention rapidly shifts to benign content. This transition induces a pronounced change in the image embedding, whereas embeddings of clean images evolve more smoothly across masking progress. BackdoorIDS operationalizes this signal by extracting an embedding sequence along the masking trajectory and applying density-based clustering such as DBSCAN. An input is flagged as backdoored if its embedding sequence forms more than one cluster. Extensive experiments show that BackdoorIDS consistently outperforms existing defenses across diverse attack types, datasets, and model families. Notably, it is a plug-and-play approach that requires no retraining and operates fully zero-shot at inference time, making it compatible with a wide range of encoder architectures, including CNNs, ViTs, CLIP, and LLaVA-1.5.
Abstract (translated)
自监督和多模态视觉编码器学习到的强大视觉表示在下游视觉任务和大型视觉语言模型（LVLM）中被广泛采用。然而，下游用户经常依赖来源不明的第三方预训练编码器，这使他们容易受到后门攻击。在这项工作中，我们提出了BackdoorIDS，这是一种简单而有效的零样本、推理时检测预训练视觉编码器中的后门样本的方法。 BackdoorIDS的设计动机来自两个观察结果：注意力劫持和恢复。在逐步输入屏蔽过程中，含有恶意触发特征的被篡改图像最初会集中注意于这些恶意特征上。一旦屏蔽比例超过了触发特征的鲁棒性阈值，该触发特征就会失效，注意力迅速转移到正常的内容上。这一转变会在图像嵌入中引起显著的变化，而干净图像的嵌入在屏蔽过程中则变化较为平缓。 BackdoorIDS通过沿着屏蔽轨迹提取一个嵌入序列并应用基于密度的聚类（如DBSCAN）来实现这一点。如果某个输入样本的嵌入序列形成了超过一个簇，则将其标记为被篡改。广泛的实验表明，无论攻击类型、数据集还是模型家族如何，BackdoorIDS都能持续超越现有的防御方法。 特别值得注意的是，BackdoorIDS是一个即插即用的方法，不需要重新训练，并且能够在推理时完全零样本操作，使其兼容包括CNNs、ViTs、CLIP和LLaVA-1.5在内的各种编码器架构。
URL
https://arxiv.org/abs/2603.11664
PDF
-
UNet-AF: An alias-free UNet for image restoration
2026-03-11 21:34:43J\'er\'emy Scanvic, Quentin Barth\'elemy, Juli\'an TachellaAbstract
The simplicity and effectiveness of the UNet architecture makes it ubiquitous in image restoration, image segmentation, and diffusion models. They are often assumed to be equivariant to translations, yet they traditionally consist of layers that are known to be prone to aliasing, which hinders their equivariance in practice. To overcome this limitation, we propose a new alias-free UNet designed from a careful selection of state-of-the-art translation-equivariant layers. We evaluate the proposed equivariant architecture against non-equivariant baselines on image restoration tasks and observe competitive performance with a significant increase in measured equivariance. Through extensive ablation studies, we also demonstrate that each change is crucial for its empirical equivariance. Our implementation is available at this https URL
Abstract (translated)
UNet架构的简单性和有效性使其在图像修复、图像分割和扩散模型中广泛使用。人们通常认为它们对平移具有等变性，但传统上却由易受混叠影响的层组成，这实际上阻碍了其等变性能。为克服这一限制，我们提出了一种新的无混叠UNet架构，该架构通过精心选择最先进的平移等变层构建而成。我们在图像修复任务中评估了所提出的等变架构，并观察到与非等变基线相比，其实验结果在等变性方面有显著提高，同时性能也具有竞争力。通过广泛的消融研究，我们也证明每个改变对实验中的等变性至关重要。我们的实现代码可在此网址获得：[此URL]（请将“this https URL”替换为实际提供的具体链接）。
URL
https://arxiv.org/abs/2603.11323
PDF
-
Frequency-Modulated Visual Restoration for Matryoshka Large Multimodal Models
2026-03-11 18:33:52Qingtao Pan, Zhihao Dou, Shuo LiAbstract
Large Multimodal Models (LMMs) struggle to adapt varying computational budgets due to numerous visual tokens. Previous methods attempted to reduce the number of visual tokens before or within LLMs. However, these strategies inevitably result in the loss of visual semantic. To address these issues, we introduce FMVR, a plug-and-play and extremely simple Frequency-Modulated Visual Restoration strategy to boost the reasoning ability of LMMs under visual token reduction. Specifically, FMVR disentangles the visual representation of fewer visual tokens into low- and high-frequency components through AvgPool and MaxPool. The derived frequencies are subsequently modulated using lightweight learnable parameters. The high-frequency from AvgPool acts as a saliency filter to enhance saliency visual semantics, while the low-frequency from MaxPool acts as an anti-saliency filter to strengthen weak visual semantics. It enables the preservation of visual semantics dominated by few visual tokens and the restoration of diluted visual semantics. Additionally, we inject FMVR into Matryoshka Representation Learning to learn coarse-to-fine visual token sets, thus enabling to elastically adjust the number of visual tokens during inference while maintaining comparable performance. Experiments across 10 image-based and 4 video-based bench marks demonstrate that FMVR-LLaVA reduce the FLOPs of LLaVA-1.5-7B by 89%, while maintaining almost 100% of the original accuracy. The code will be open.
Abstract (translated)
大型多模态模型（LMMs）在适应不同的计算预算时面临挑战，这主要是由于存在大量的视觉标记。之前的方法试图通过减少输入到LLMs之前的视觉标记数量来解决这个问题。然而，这些策略不可避免地会导致视觉语义的丢失。为了解决这些问题，我们引入了FMVR（Frequency-Modulated Visual Restoration），这是一种插件式且极其简单的策略，旨在增强在视觉标记减少情况下的LMM推理能力。 具体来说，FMVR通过使用AvgPool和MaxPool将较少视觉标记的视觉表示分解为低频和高频分量。随后，利用轻量级可学习参数对这些频率进行调制。来自AvgPool的高频部分充当了显著性过滤器，用于增强显著性的视觉语义；而来自MaxPool的低频部分则充当非显著性过滤器，用来加强弱化的视觉语义。这使得在使用少量视觉标记的情况下保留主导视觉语义，并恢复稀释的视觉语义成为可能。 此外，我们将FMVR整合到Matryoshka表示学习中，以从粗到细地学习视觉令牌集，从而能够在推理过程中灵活调整视觉令牌的数量，同时保持与原模型相当的表现力。跨10个基于图像和4个基于视频的基准测试的实验表明，在减少LLaVA-1.5-7B的FLOPs（89%）的同时，FMVR-LLaVA能够几乎维持原始准确性的100%不变。代码将会公开。
URL
https://arxiv.org/abs/2603.11220
PDF
-
UHD Image Deblurring via Autoregressive Flow with Ill-conditioned Constraints
2026-03-11 08:19:34Yucheng Xin, Dawei Zhao, Xiang Chen, Chen Wu, Pu Wang, Dianjie Lu, Guijuan Zhang, Xiuyi Jia, Zhuoran ZhengAbstract
Ultra-high-definition (UHD) image deblurring poses significant challenges for UHD restoration methods, which must balance fine-grained detail recovery and practical inference efficiency. Although prominent discriminative and generative methods have achieved remarkable results, a trade-off persists between computational cost and the ability to generate fine-grained detail for UHD image deblurring tasks. To further alleviate these issues, we propose a novel autoregressive flow method for UHD image deblurring with an ill-conditioned constraint. Our core idea is to decompose UHD restoration into a progressive, coarse-to-fine process: at each scale, the sharp estimate is formed by upsampling the previous-scale result and adding a current-scale residual, enabling stable, stage-wise refinement from low to high resolution. We further introduce Flow Matching to model residual generation as a conditional vector field and perform few-step ODE sampling with efficient Euler/Heun solvers, enriching details while keeping inference affordable. Since multi-step generation at UHD can be numerically unstable, we propose an ill-conditioning suppression scheme by imposing condition-number regularization on a feature-induced attention matrix, improving convergence and cross-scale consistency. Our method demonstrates promising performance on blurred images at 4K (3840$\times$2160) or higher resolutions.
Abstract (translated)
超高清（UHD）图像去模糊对恢复方法提出了重大挑战，这些方法必须在精细细节恢复和实用推理效率之间取得平衡。尽管一些显著的判别性和生成性方法已经取得了令人瞩目的成果，在计算成本与生成超高清图像去模糊任务中细粒度细节的能力之间仍存在权衡问题。为了解决这些问题，我们提出了一种新的自回归流方法用于带有病态条件约束的UHD图像去模糊。 我们的核心思想是将UHD恢复过程分解成一个渐进式的、由粗到细的过程：在每一尺度下，锐化估计通过上采样前一尺度的结果并添加当前尺度的残差来形成，从而实现从低分辨率到高分辨率的稳定阶段式细化。我们进一步引入流匹配（Flow Matching）以将残差生成建模为条件向量场，并采用高效的欧拉/赫恩求解器进行少量步骤的常微分方程采样，在保持推理成本可控的同时丰富细节。 由于在UHD情况下多步生成可能会出现数值不稳定性，我们提出了一种病态抑制方案，通过对特征诱导注意力矩阵施加条件数正则化来改善收敛性和跨尺度一致性。我们的方法在4K（3840x2160）或更高分辨率的模糊图像上表现出色。
URL
https://arxiv.org/abs/2603.10517
PDF
-
Decoder-Free Distillation for Quantized Image Restoration
2026-03-10 13:03:47S. M. A. Sharif, Abdur Rehman, Seongwan Kim, Jaeho LeeAbstract
Quantization-Aware Training (QAT), combined with Knowledge Distillation (KD), holds immense promise for compressing models for edge deployment. However, joint optimization for precision-sensitive image restoration (IR) to recover visual quality from degraded images remains largely underexplored. Directly adapting QAT-KD to low-level vision reveals three critical bottlenecks: teacher-student capacity mismatch, spatial error amplification during decoder distillation, and an optimization "tug-of-war" between reconstruction and distillation losses caused by quantization noise. To tackle these, we introduce Quantization-aware Distilled Restoration (QDR), a framework for edge-deployed IR. QDR eliminates capacity mismatch via FP32 self-distillation and prevents error amplification through Decoder-Free Distillation (DFD), which corrects quantization errors strictly at the network bottleneck. To stabilize the optimization tug-of-war, we propose a Learnable Magnitude Reweighting (LMR) that dynamically balances competing gradients. Finally, we design an Edge-Friendly Model (EFM) featuring a lightweight Learnable Degradation Gating (LDG) to dynamically modulate spatial degradation localization. Extensive experiments across four IR tasks demonstrate that our Int8 model recovers 96.5% of FP32 performance, achieves 442 frames per second (FPS) on an NVIDIA Jetson Orin, and boosts downstream object detection by 16.3 mAP
Abstract (translated)
量化感知训练（QAT）与知识蒸馏（KD）的结合为模型压缩以便在边缘设备上部署带来了巨大的潜力。然而，针对对精度敏感的图像恢复（IR），即从退化图像中恢复视觉质量的整体优化问题仍然很大程度上未被探索。直接将QAT-KD应用于低级视觉揭示了三个关键瓶颈：教师-学生容量不匹配、解码器蒸馏过程中的空间误差放大以及由于量化噪声导致重建和蒸馏损失之间的“拉锯战”。为了应对这些问题，我们引入了量化的蒸馏恢复（QDR），这是一种面向边缘部署的IR框架。QDR通过FP32自我蒸馏消除容量差异，并通过无解码器蒸馏（DFD）防止误差放大，严格纠正网络瓶颈处的量化错误。为稳定优化“拉锯战”，我们提出了一种可学习幅度重新加权（LMR），动态地平衡竞争梯度。最后，我们设计了一个友好的边缘模型（EFM），该模型具有轻量级的学习退化门控（LDG）以动态调节空间退化定位。跨四个IR任务的广泛实验表明，我们的Int8模型恢复了96.5%的FP32性能，在NVIDIA Jetson Orin上实现了442帧每秒（FPS），并且提高了下游目标检测的16.3 mAP。 这种QDR框架通过引入一系列创新的技术和策略来克服传统方法中的瓶颈，显著提高了压缩模型在边缘设备上的部署效率与效果。
URL
https://arxiv.org/abs/2603.09624
PDF
-
Component-Aware Sketch-to-Image Generation Using Self-Attention Encoding and Coordinate-Preserving Fusion
2026-03-10 10:39:24Ali Zia, Muhammad Umer Ramzan, Usman Ali, Muhammad Faheem, Abdelwahed Khamis, Shahnawaz QureshiAbstract
Translating freehand sketches into photorealistic images remains a fundamental challenge in image synthesis, particularly due to the abstract, sparse, and stylistically diverse nature of sketches. Existing approaches, including GAN-based and diffusion-based models, often struggle to reconstruct fine-grained details, maintain spatial alignment, or adapt across different sketch domains. In this paper, we propose a component-aware, self-refining framework for sketch-to-image generation that addresses these challenges through a novel two-stage architecture. A Self-Attention-based Autoencoder Network (SA2N) first captures localised semantic and structural features from component-wise sketch regions, while a Coordinate-Preserving Gated Fusion (CGF) module integrates these into a coherent spatial layout. Finally, a Spatially Adaptive Refinement Revisor (SARR), built on a modified StyleGAN2 backbone, enhances realism and consistency through iterative refinement guided by spatial context. Extensive experiments across both facial (CelebAMask-HQ, CUFSF) and non-facial (Sketchy, ChairsV2, ShoesV2) datasets demonstrate the robustness and generalizability of our method. The proposed framework consistently outperforms state-of-the-art GAN and diffusion models, achieving significant gains in image fidelity, semantic accuracy, and perceptual quality. On CelebAMask-HQ, our model improves over prior methods by 21% (FID), 58% (IS), 41% (KID), and 20% (SSIM). These results, along with higher efficiency and visual coherence across diverse domains, position our approach as a strong candidate for applications in forensics, digital art restoration, and general sketch-based image synthesis.
Abstract (translated)
将手绘草图转换为逼真的图像仍然是图像合成中的一个基本挑战，尤其是因为草图具有抽象、稀疏和风格多样的特点。现有的方法，包括基于GAN的模型和扩散模型，往往难以重建细粒度细节、保持空间对齐或适应不同的草图领域。在本文中，我们提出了一种组件感知、自我完善的框架，用于解决从草图生成图像时遇到的问题，并通过新颖的两阶段架构来克服这些挑战。首先，基于自注意力机制的自动编码网络（Self-Attention-based Autoencoder Network, SA2N）捕获来自按组件划分的草图区域的局部语义和结构特征；然后，坐标保持门控融合模块（Coordinate-Preserving Gated Fusion, CGF）将这些信息整合成一个连贯的空间布局。最后，基于修改后的StyleGAN2骨干网构建的空间自适应细化修订器（Spatially Adaptive Refinement Revisor, SARR），通过空间上下文引导的迭代改进来增强现实性和一致性。 在面部（CelebAMask-HQ、CUFSF）和非面部（Sketchy、ChairsV2、ShoesV2）数据集上的广泛实验验证了我们方法的鲁棒性与泛化能力。我们的框架持续优于最先进的GAN和扩散模型，在图像保真度、语义准确性及感知质量方面取得了显著提升。在CelebAMask-HQ上，我们的模型相比先前的方法分别提高了21%（FID）、58%（IS）、41%（KID）以及20%（SSIM）。这些成果，加上更高的效率和跨不同领域的视觉连贯性，使我们的方法成为法医学、数字艺术品修复及通用草图生成图像合成应用的有力候选。
URL
https://arxiv.org/abs/2603.09484
PDF
-
InfiniteDance: Scalable 3D Dance Generation Towards in-the-wild Generalization
2026-03-10 10:10:34Ronghui Li, Zhongyuan Hu, Li Siyao, Youliang Zhang, Haozhe Xie, Mingyuan Zhang, Jie Guo, Xiu Li, Ziwei LiuAbstract
Although existing 3D dance generation methods perform well in controlled scenarios, they often struggle to generalize in the wild. When conditioned on unseen music, existing methods often produce unstructured or physically implausible dance, largely due to limited music-to-dance data and restricted model capacity. This work aims to push the frontier of generalizable 3D dance generation by scaling up both data and model design. (1) On the data side, we develop a fully automated pipeline that reconstructs high-fidelity 3D dance motions from monocular videos. To eliminate the physical artifacts prevalent in existing reconstruction methods, we introduce a Foot Restoration Diffusion Model (FRDM) guided by foot-contact and geometric constraints that enforce physical plausibility while preserving kinematic smoothness and expressiveness, resulting in a diverse, high-quality multimodal 3D dance dataset totaling 100.69 hours. (2) On model design, we propose Choreographic LLaMA (ChoreoLLaMA), a scalable LLaMA-based architecture. To enhance robustness under unfamiliar music conditions, we integrate a retrieval-augmented generation (RAG) module that injects reference dance as a prompt. Additionally, we design a slow/fast-cadence Mixture-of-Experts (MoE) module that enables ChoreoLLaMA to smoothly adapt motion rhythms across varying music tempos. Extensive experiments across diverse dance genres show that our approach surpasses existing methods in both qualitative and quantitative evaluations, marking a step toward scalable, real-world 3D dance generation. Code, models, and data will be released.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13375
PDF
