Abstract
Feed-forward 3D Gaussian Splatting (3DGS) models enable real-time scene generation but are hindered by suboptimal pixel-aligned primitive placement, which relies on a dense, rigid grid and limits both quality and efficiency. We introduce a new feed-forward architecture that detects 3D Gaussian primitives at a sub-pixel level, replacing the pixel grid with an adaptive, "Off The Grid" distribution. Inspired by keypoint detection, our multi-resolution decoder learns to distribute primitives across image patches. This module is trained end-to-end with a 3D reconstruction backbone using self-supervised learning. Our resulting pose-free model generates photorealistic scenes in seconds, achieving state-of-the-art novel view synthesis for feed-forward models. It outperforms competitors while using far fewer primitives, demonstrating a more accurate and efficient allocation that captures fine details and reduces artifacts. Moreover, we observe that by learning to render 3D Gaussians, our 3D reconstruction backbone improves camera pose estimation, suggesting opportunities to train these foundational models without labels.
Abstract (translated)
前馈3D高斯点阵模型（3DGS）能够实现实时场景生成，但由于像素对齐的基本元素放置不理想而受限。这种限制依赖于密集且刚性的网格结构，从而影响了质量和效率的提升。我们引入了一种新的前馈架构，该架构能够在亚像素级别检测3D高斯基本体，并用一种自适应、"离网"(Off The Grid)分布取代传统的像素网格。受关键点检测的启发，我们的多分辨率解码器能够学习在图像补丁中分配这些基本元素。这一模块与用于三维重建的骨干网络一起，使用自我监督学习方法进行端到端训练。 我们得到的结果是没有姿态约束的模型可以在几秒钟内生成逼真的场景，并且在前馈模型的新视角合成方面达到了最先进的水平。它不仅超越了竞争对手，在使用远少于其他模型的基本体数量的同时，展示了一种更为准确和高效的分配方式，该方法能够捕捉到细节并减少伪影。此外，我们观察到通过学习渲染3D高斯分布，我们的三维重建骨干网络可以提高相机姿态估计的准确性，这表明有可能在没有标签的情况下训练这些基础模型。
URL
https://arxiv.org/abs/2512.15508
