Abstract
Robotic manipulation requires anticipating how the environment evolves in response to actions, yet most existing systems lack this predictive capability, often resulting in errors and inefficiency. While Vision-Language Models (VLMs) provide high-level guidance, they cannot explicitly forecast future states, and existing world models either predict only short horizons or produce spatially inconsistent frames. To address these challenges, we propose a framework for fast and predictive video-conditioned action. Our approach first selects and adapts a robust video generation model to ensure reliable future predictions, then applies adversarial distillation for fast, few-step video generation, and finally trains an action model that leverages both generated videos and real observations to correct spatial errors. Extensive experiments show that our method produces temporally coherent, spatially accurate video predictions that directly support precise manipulation, achieving significant improvements in embodiment consistency, spatial referring ability, and task completion over existing baselines. Codes & Models will be released.
Abstract (translated)
机器人操作需要预测环境在行动后的演变情况，然而大多数现有系统缺乏这种预测能力，导致经常出现错误和效率低下。虽然视觉-语言模型（VLMs）提供了高层次的指导，但它们无法明确地预测未来的状态，而现有的世界模型要么只能预测短期的时间范围，要么生成的空间上不一致的画面。为了解决这些问题，我们提出了一种快速且具备预测能力的视频条件动作框架。 我们的方法首先选择并调整一个稳健的视频生成模型，以确保可靠的未来预测；然后应用对抗蒸馏（adversarial distillation）进行快速、少步骤的视频生成；最后训练一个利用生成的视频和真实观察来纠正空间错误的动作模型。大量的实验表明，我们提出的方法能够产生时间上连贯且空间上准确的视频预测，直接支持精确的操作，并在具身一致性（embodiment consistency）、空间指代能力以及任务完成度方面显著优于现有的基准方法。 代码与模型将在适当的时候发布。
URL
https://arxiv.org/abs/2602.10717
