Abstract
Face image restoration aims to enhance degraded facial images while addressing challenges such as diverse degradation types, real-time processing demands, and, most crucially, the preservation of identity-specific features. Existing methods often struggle with slow processing times and suboptimal restoration, especially under severe degradation, failing to accurately reconstruct finer-level identity details. To address these issues, we introduce InstantRestore, a novel framework that leverages a single-step image diffusion model and an attention-sharing mechanism for fast and personalized face restoration. Additionally, InstantRestore incorporates a novel landmark attention loss, aligning key facial landmarks to refine the attention maps, enhancing identity preservation. At inference time, given a degraded input and a small (~4) set of reference images, InstantRestore performs a single forward pass through the network to achieve near real-time performance. Unlike prior approaches that rely on full diffusion processes or per-identity model tuning, InstantRestore offers a scalable solution suitable for large-scale applications. Extensive experiments demonstrate that InstantRestore outperforms existing methods in quality and speed, making it an appealing choice for identity-preserving face restoration.
Abstract (translated)
面部图像修复旨在增强退化的面部图像，同时应对多样化的退化类型、实时处理需求以及最重要的是保持特定身份特征的挑战。现有方法常常在处理时间慢和恢复效果不佳方面存在问题，尤其是在严重退化的情况下，无法准确重建更细粒度的身份细节。为了解决这些问题，我们引入了InstantRestore，这是一个新型框架，利用单步图像扩散模型和注意力共享机制实现快速且个性化的面部修复。此外，InstantRestore集成了一个新颖的地标注意损失函数，通过关键面部地标的对齐来优化注意力图，增强身份特征的保持。在推理过程中，给定退化输入和一组较小（约4张）参考图像，InstantRestore只需一次前向传递即可实现接近实时性能。与依赖完整扩散过程或针对每个个体模型调优的传统方法不同，InstantRestore提供了一个适合大规模应用的可扩展解决方案。广泛的实验表明，InstantRestore在质量和速度上均优于现有方法，使其成为保持身份特征面部修复的一个有吸引力的选择。
URL
https://arxiv.org/abs/2412.06753
