Abstract
Robotic arm manipulation in data-scarce settings is a highly challenging task due to the complex embodiment dynamics and diverse contexts. Recent video-based approaches have shown great promise in capturing and transferring the temporal and physical interactions by pre-training on Internet-scale video data. However, such methods are often not optimized for the embodiment-specific closed-loop control, typically suffering from high latency and insufficient grounding. In this paper, we present Vidarc (Video Diffusion for Action Reasoning and Closed-loop Control), a novel autoregressive embodied video diffusion approach augmented by a masked inverse dynamics model. By grounding video predictions with action-relevant masks and incorporating real-time feedback through cached autoregressive generation, Vidarc achieves fast, accurate closed-loop control. Pre-trained on one million cross-embodiment episodes, Vidarc surpasses state-of-the-art baselines, achieving at least a 15% higher success rate in real-world deployment and a 91% reduction in latency. We also highlight its robust generalization and error correction capabilities across previously unseen robotic platforms.
Abstract (translated)
在数据匮乏的环境中进行机器人手臂操作是一项极具挑战性的任务，因为需要处理复杂的物理动力学和多种多样的场景。近期基于视频的方法通过在互联网规模的视频数据上预训练，展现出了捕捉并转移时序与物理交互的强大潜力。然而，这些方法通常并未针对特定执行体（如机器人臂）的闭环控制进行优化，往往存在高延迟和不够精确的问题。 本文介绍了Vidarc（Video Diffusion for Action Reasoning and Closed-loop Control），一种新颖的基于视频扩散的自回归式具身化方法，并通过加入掩码逆动力学模型得到增强。该方法通过使用与动作相关的掩模来将视频预测与实际操作关联起来，同时利用缓存的自回归生成方式实时反馈调整，实现了快速且精准的闭环控制。经过针对不同执行体百万级别的数据集预训练后，Vidarc在真实环境中的部署中，相较于现有最佳基准模型成功率提高了至少15%，并且延迟降低了91%。此外，本文还强调了其在未见过的新机器人平台上的鲁棒性泛化和错误校正能力。 这一技术的进步为解决机器人手臂操作领域内的数据不足问题提供了一种新的有效途径，并展示了自回归视频扩散方法在未来具身智能系统中的巨大潜力。
URL
https://arxiv.org/abs/2512.17661
