Abstract
This paper introduces Multi-Garment Customized Model Generation, a unified framework based on Latent Diffusion Models (LDMs) aimed at addressing the unexplored task of synthesizing images with free combinations of multiple pieces of clothing. The method focuses on generating customized models wearing various targeted outfits according to different text prompts. The primary challenge lies in maintaining the natural appearance of the dressed model while preserving the complex textures of each piece of clothing, ensuring that the information from different garments does not interfere with each other. To tackle these challenges, we first developed a garment encoder, which is a trainable UNet copy with shared weights, capable of extracting detailed features of garments in parallel. Secondly, our framework supports the conditional generation of multiple garments through decoupled multi-garment feature fusion, allowing multiple clothing features to be injected into the backbone network, significantly alleviating conflicts between garment information. Additionally, the proposed garment encoder is a plug-and-play module that can be combined with other extension modules such as IP-Adapter and ControlNet, enhancing the diversity and controllability of the generated models. Extensive experiments demonstrate the superiority of our approach over existing alternatives, opening up new avenues for the task of generating images with multiple-piece clothing combinations
Abstract (translated)
本文介绍了一种名为多件定制模型生成（Multi-Garment Customized Model Generation）的统一框架，基于潜在扩散模型（LDMs），旨在解决使用多种服装组合生成图像的未解决任务。该方法专注于根据不同的文本提示生成各种目标服装的定制模型。主要的挑战在于保持穿着模型的自然外观，同时保留每件服装的复杂纹理，确保不同服装的信息不互相干扰。为解决这些挑战，我们首先开发了一个服装编码器，是一个可训练的UNet副本，具有共享权重，能够并行提取服装的详细特征。其次，我们的框架通过解耦多件服装特征融合支持了 multiple garments 的条件生成，允许将多个服装特征注入到骨干网络中，显著减轻了服装信息之间的冲突。此外，所提出的服装编码器是一个可插拔的模块，可以与其他扩展模块如IP-Adapter和ControlNet 结合使用，提高生成的模型的多样性和可控性。大量实验证明，我们的方法相对于现有方法具有优越性，为生成多件服装组合的图像开辟了新的途径。
URL
https://arxiv.org/abs/2408.05206
