Abstract
Recent advances in driving-scene generation and reconstruction have demonstrated significant potential for enhancing autonomous driving systems by producing scalable and controllable training data. Existing generation methods primarily focus on synthesizing diverse and high-fidelity driving videos; however, due to limited 3D consistency and sparse viewpoint coverage, they struggle to support convenient and high-quality novel-view synthesis (NVS). Conversely, recent 3D/4D reconstruction approaches have significantly improved NVS for real-world driving scenes, yet inherently lack generative capabilities. To overcome this dilemma between scene generation and reconstruction, we propose \textbf{WorldSplat}, a novel feed-forward framework for 4D driving-scene generation. Our approach effectively generates consistent multi-track videos through two key steps: ((i)) We introduce a 4D-aware latent diffusion model integrating multi-modal information to produce pixel-aligned 4D Gaussians in a feed-forward manner. ((ii)) Subsequently, we refine the novel view videos rendered from these Gaussians using a enhanced video diffusion model. Extensive experiments conducted on benchmark datasets demonstrate that \textbf{WorldSplat} effectively generates high-fidelity, temporally and spatially consistent multi-track novel view driving videos.
Abstract (translated)
最近在驾驶场景生成和重建方面的进展展示了其增强自动驾驶系统的能力，通过生产可扩展且可控的训练数据。现有的生成方法主要集中在合成多样性和高保真的驾驶视频上；然而，由于三维一致性有限和视角覆盖稀疏的问题，它们难以支持便捷、高质量的新视点合成（NVS）。相比之下，最近的3D/4D重建方法在真实世界的驾驶场景中显著提高了NVS的质量，但天生缺乏生成能力。为了克服这种在场景生成与重建之间的困境，我们提出了**WorldSplat**，这是一种新颖的前馈框架，用于生成四维驾驶场景。我们的方法通过两个关键步骤有效地产生了具有一致性的多轨道视频：（i）我们引入了一种4D感知的潜在扩散模型，该模型集成了多模态信息，并以前馈方式生成像素对齐的4D高斯分布。（ii）随后，利用增强的视频扩散模型来细化从这些高斯分布渲染出的新视点视频。在基准数据集上进行的一系列实验表明，**WorldSplat**能够有效生成高度保真、时间和空间一致性的多轨道新视点驾驶视频。
URL
https://arxiv.org/abs/2509.23402
