Abstract
Indoor scene texture synthesis has garnered significant interest due to its important potential applications in virtual reality, digital media, and creative arts. Existing diffusion model-based researches either rely on per-view inpainting techniques, which are plagued by severe cross-view inconsistencies and conspicuous seams, or they resort to optimization-based approaches that entail substantial computational overhead. In this work, we present RoomPainter, a framework that seamlessly integrates efficiency and consistency to achieve high-fidelity texturing of indoor scenes. The core of RoomPainter features a zero-shot technique that effectively adapts a 2D diffusion model for 3D-consistent texture synthesis, along with a two-stage generation strategy that ensures both global and local consistency. Specifically, we introduce Attention-Guided Multi-View Integrated Sampling (MVIS) combined with a neighbor-integrated attention mechanism for zero-shot texture map generation. Using the MVIS, we firstly generate texture map for the entire room to ensure global consistency, then adopt its variant, namely an attention-guided multi-view integrated repaint sampling (MVRS) to repaint individual instances within the room, thereby further enhancing local consistency. Experiments demonstrate that RoomPainter achieves superior performance for indoor scene texture synthesis in visual quality, global consistency, and generation efficiency.
Abstract (translated)
室内场景纹理合成因其在虚拟现实、数字媒体和创意艺术中的重要潜在应用而引起了广泛的关注。现有的基于扩散模型的研究要么依赖于视图内插技术，这些技术受到严重跨视角不一致性和明显接缝的影响，要么采用计算成本高昂的优化方法。在这项工作中，我们提出了RoomPainter框架，该框架无缝集成了效率和一致性，以实现室内场景的高质量纹理化。RoomPainter的核心是一个零样本技术，能够有效地将2D扩散模型适应于3D一致性的纹理合成，并结合两阶段生成策略确保全局和局部的一致性。具体而言，我们引入了基于注意引导的多视图集成采样（MVIS）技术，以及一个邻居集成注意力机制来实现零样本纹理贴图生成。利用MVIS，我们首先为整个房间生成纹理贴图以确保全局一致性，然后采用其变体——注意引导的多视图再绘制采样（MVRS），重新绘制房间内的各个实例，从而进一步增强局部一致性。实验表明，RoomPainter在视觉质量、全局一致性和生成效率方面实现了室内场景纹理合成的优越性能。
URL
https://arxiv.org/abs/2412.16778
