Abstract
Video restoration in real-world scenarios is challenged by heterogeneous degradations, where static architectures and fixed inference pipelines often fail to generalize. Recent agent-based approaches offer dynamic decision making, yet existing video restoration agents remain limited by insufficient quality perception and inefficient search strategies. We propose VQ-Jarvis, a retrieval-augmented, all-in-one intelligent video restoration agent with sharper vision and faster thought. VQ-Jarvis is designed to accurately perceive degradations and subtle differences among paired restoration results, while efficiently discovering optimal restoration trajectories. To enable sharp vision, we construct VSR-Compare, the first large-scale video paired enhancement dataset with 20K comparison pairs covering 7 degradation types, 11 enhancement operators, and diverse content domains. Based on this dataset, we train a multiple operator judge model and a degradation perception model to guide agent decisions. To achieve fast thought, we introduce a hierarchical operator scheduling strategy that adapts to video difficulty: for easy cases, optimal restoration trajectories are retrieved in a one-step manner from a retrieval-augmented generation (RAG) library; for harder cases, a step-by-step greedy search is performed to balance efficiency and accuracy. Extensive experiments demonstrate that VQ-Jarvis consistently outperforms existing methods on complex degraded videos.
Abstract (translated)
在真实场景中，视频修复面临异质退化的挑战，静态架构与固定推理流程往往难以泛化应对。近期基于智能体的方法虽能实现动态决策，但现有视频修复智能体仍受限于质量感知不足与搜索策略低效。我们提出VQ-Jarvis——一个检索增强型全能智能视频修复系统，具备更敏锐的视觉感知与更迅捷的决策能力。该系统可精准识别退化类型及配对修复结果间的细微差异，同时高效探索最优修复路径。为实现敏锐视觉，我们构建了VSR-Compare——首个大规模视频配对增强数据集，包含2万组对比样本，涵盖7类退化类型、11种增强算子及多样化的内容领域。基于该数据集，我们训练了多算子评判模型与退化感知模型以指导智能体决策。为达成快速推理，我们引入分层算子调度策略，可根据视频难度自适应调整：对简单案例，通过检索增强生成（RAG）库实现单步检索最优修复路径；对复杂案例则采用逐步贪心搜索，在效率与精度间取得平衡。大量实验表明，VQ-Jarvis在复杂退化视频场景下持续优于现有方法。
URL
https://arxiv.org/abs/2603.22998
