Abstract
Recent advances in diffusion transformers have empowered video generation models to generate high-quality video clips from texts or images. However, world models with the ability to predict long-horizon futures from past observations and actions remain underexplored, especially for general-purpose scenarios and various forms of actions. To bridge this gap, we introduce Astra, an interactive general world model that generates real-world futures for diverse scenarios (e.g., autonomous driving, robot grasping) with precise action interactions (e.g., camera motion, robot action). We propose an autoregressive denoising architecture and use temporal causal attention to aggregate past observations and support streaming outputs. We use a noise-augmented history memory to avoid over-reliance on past frames to balance responsiveness with temporal coherence. For precise action control, we introduce an action-aware adapter that directly injects action signals into the denoising process. We further develop a mixture of action experts that dynamically route heterogeneous action modalities, enhancing versatility across diverse real-world tasks such as exploration, manipulation, and camera control. Astra achieves interactive, consistent, and general long-term video prediction and supports various forms of interactions. Experiments across multiple datasets demonstrate the improvements of Astra in fidelity, long-range prediction, and action alignment over existing state-of-the-art world models.
Abstract (translated)
最近的扩散变换器进展使视频生成模型能够从文本或图像中生成高质量的视频片段。然而，具有预测长期未来的能力的世界模型（基于过去观察和行动）在一般用途场景以及各种形式的动作方面仍然较少被探索。为了填补这一空白，我们引入了Astra，这是一个交互式通用世界模型，它为多样化的场景（如自动驾驶、机器人抓取等）生成现实世界的未来，并能够进行精确的动作互动（如相机移动、机器人动作）。我们提出了一个自回归去噪架构，并使用时间因果注意力机制来聚合过去的观察结果并支持流输出。为了不依赖于过去的帧而过度，我们采用了一种噪声增强的历史记忆方法，以平衡响应性与时间一致性。对于精确的动作控制，我们引入了一个感知动作的适配器，它可以直接将动作信号注入去噪过程中。此外，我们开发了一组行动专家混合模型，这些模型能够动态地路由不同形式的动作模态，增强了在探索、操作和相机控制等多样化现实世界任务中的适应性。Astra实现了交互式的长期视频预测，并支持多种形式的互动。跨多个数据集的实验显示了Astra在精确度、长范围预测及动作对齐方面相对于现有最先进的世界模型的进步。
URL
https://arxiv.org/abs/2512.08931
