Abstract
Recent text-to-video models have demonstrated strong temporal generation capabilities, yet their potential for image restoration remains underexplored. In this work, we repurpose CogVideo for progressive visual restoration tasks by fine-tuning it to generate restoration trajectories rather than natural video motion. Specifically, we construct synthetic datasets for super-resolution, deblurring, and low-light enhancement, where each sample depicts a gradual transition from degraded to clean frames. Two prompting strategies are compared: a uniform text prompt shared across all samples, and a scene-specific prompting scheme generated via LLaVA multi-modal LLM and refined with ChatGPT. Our fine-tuned model learns to associate temporal progression with restoration quality, producing sequences that improve perceptual metrics such as PSNR, SSIM, and LPIPS across frames. Extensive experiments show that CogVideo effectively restores spatial detail and illumination consistency while maintaining temporal coherence. Moreover, the model generalizes to real-world scenarios on the ReLoBlur dataset without additional training, demonstrating strong zero-shot robustness and interpretability through temporal restoration.
Abstract (translated)
最近的文本到视频模型展示了强大的时间生成能力，但它们在图像恢复方面的潜力尚未得到充分探索。在这项工作中，我们将CogVideo重新用于渐进式视觉恢复任务，通过微调使其生成恢复轨迹而非自然视频运动。具体来说，我们构建了超分辨率、去模糊和低光增强的合成数据集，在这些数据集中，每个样本都展示了从降质图像到清晰图像的逐步过渡。我们比较了两种提示策略：一种是在所有样本中共享的统一文本提示，另一种是通过LLaVA多模态大语言模型生成并由ChatGPT优化的情境特定提示方案。 经过微调后，我们的模型学会了将时间进程与恢复质量关联起来，在帧之间改善如峰值信噪比（PSNR）、结构相似性指数度量（SSIM）和感知线性判别分析（LPIPS）等感知指标。广泛的实验表明，CogVideo有效地恢复了空间细节和光照一致性，并保持了时间连贯性。此外，该模型在无需额外训练的情况下，在ReLoBlur数据集上的真实场景中表现出强大的零样本鲁棒性和通过时间恢复的可解释性。
URL
https://arxiv.org/abs/2512.02273
