Abstract
We introduce MAGI, a hybrid video generation framework that combines masked modeling for intra-frame generation with causal modeling for next-frame generation. Our key innovation, Complete Teacher Forcing (CTF), conditions masked frames on complete observation frames rather than masked ones (namely Masked Teacher Forcing, MTF), enabling a smooth transition from token-level (patch-level) to frame-level autoregressive generation. CTF significantly outperforms MTF, achieving a +23% improvement in FVD scores on first-frame conditioned video prediction. To address issues like exposure bias, we employ targeted training strategies, setting a new benchmark in autoregressive video generation. Experiments show that MAGI can generate long, coherent video sequences exceeding 100 frames, even when trained on as few as 16 frames, highlighting its potential for scalable, high-quality video generation.
Abstract (translated)
我们介绍了MAGI，这是一个结合了掩码建模（用于帧内生成）和因果建模（用于下一帧生成）的混合视频生成框架。我们的关键创新是“完整教师强制”（Complete Teacher Forcing，CTF），它使掩码帧依赖于完整的观察帧而非仅依赖于其他掩码帧（即Masked Teacher Forcing，MTF）。这种方法使得从标记级（补丁级）到帧级的自回归生成能够平滑过渡。实验结果显示，CTF显著优于MTF，在第一帧条件下的视频预测中，FVD评分提高了23%。 为了应对诸如曝光偏差等问题，我们采用了有针对性的训练策略，并在自回归视频生成领域设立了新的基准标准。实验表明，MAGI能够在仅使用16帧进行训练的情况下，生成超过100帧长且连贯的视频序列，这突显了其在大规模、高质量视频生成中的巨大潜力。
URL
https://arxiv.org/abs/2501.12389
