Abstract
Personalized image generation has been significantly advanced, enabling the creation of highly realistic and customized images. However, existing methods often struggle with generating images of multiple people due to occlusions and fail to accurately personalize full-body shapes. In this paper, we propose PersonaCraft, a novel approach that combines diffusion models with 3D human modeling to address these limitations. Our method effectively manages occlusions by incorporating 3D-aware pose conditioning with SMPLx-ControlNet and accurately personalizes human full-body shapes through SMPLx fitting. Additionally, PersonaCraft enables user-defined body shape adjustments, adding flexibility for individual body customization. Experimental results demonstrate the superior performance of PersonaCraft in generating high-quality, realistic images of multiple individuals while resolving occlusion issues, thus establishing a new standard for multi-person personalized image synthesis. Project page: this https URL
Abstract (translated)
个性化图像生成技术已经取得了显著的进步，能够创造出高度逼真和个性化的图像。然而，现有的方法在生成多个人物的图像时常常因为遮挡问题而遇到困难，并且无法准确地个性化全身形状。在这篇论文中，我们提出了PersonaCraft，这是一种结合扩散模型与3D人体建模的新方法，以解决这些限制。我们的方法通过引入3D感知姿态条件控制（使用SMPLx-ControlNet）有效管理了遮挡问题，并通过SMPLx拟合准确地个性化了人类的全身形状。此外，PersonaCraft还允许用户定义身体形状调整，为个体的身体定制增加了灵活性。实验结果表明，PersonaCraft在生成高质量、逼真的多个人物图像方面表现出色，并解决了遮挡问题，从而确立了多人个性化图像合成的新标准。项目页面：这个 https URL
URL
https://arxiv.org/abs/2411.18068
