Abstract
Scalable generation of outdoor driving scenes requires 3D representations that remain consistent across multiple viewpoints and scale to large areas. Existing solutions either rely on image or video generative models distilled to 3D space, harming the geometric coherence and restricting the rendering to training views, or are limited to small-scale 3D scene or object-centric generation. In this work, we propose a 3D generative framework based on $\Sigma$-Voxfield grid, a discrete representation where each occupied voxel stores a fixed number of colorized surface samples. To generate this representation, we train a semantic-conditioned diffusion model that operates on local voxel neighborhoods and uses 3D positional encodings to capture spatial structure. We scale to large scenes via progressive spatial outpainting over overlapping regions. Finally, we render the generated $\Sigma$-Voxfield grid with a deferred rendering module to obtain photorealistic images, enabling large-scale multiview-consistent 3D scene generation without per-scene optimization. Extensive experiments show that our approach can generate diverse large-scale urban outdoor scenes, renderable into photorealistic images with various sensor configurations and camera trajectories while maintaining moderate computation cost compared to existing approaches.
Abstract (translated)
可扩展的户外驾驶场景生成需要具备多视角一致性和大规模扩展能力的3D表示。现有方案要么依赖蒸馏至3D空间的图像或视频生成模型，损害了几何一致性并将渲染限制在训练视角内；要么局限于小规模3D场景或以物体为中心的生成。本研究提出一种基于Σ-Voxfield网格的3D生成框架——这是一种离散表示，每个占据的体素存储固定数量的着色表面样本。为生成该表示，我们训练了一个语义条件扩散模型，该模型在局部体素邻域上运行，并使用3D位置编码捕捉空间结构。我们通过重叠区域的渐进式空间扩展绘制来实现大规模场景生成。最后，我们通过延迟渲染模块对生成的Σ-Voxfield网格进行渲染，获得逼真图像，实现了无需逐场景优化的大规模多视角一致3D场景生成。大量实验表明，我们的方法能够生成多样的大规模城市户外场景，可渲染为具有各种传感器配置和相机轨迹的逼真图像，同时相较于现有方法保持适中的计算成本。
URL
https://arxiv.org/abs/2604.06113
