Abstract
Generating images from text involving complex and novel object arrangements remains a significant challenge for current text-to-image (T2I) models. Although prior layout-based methods improve object arrangements using spatial constraints with 2D layouts, they often struggle to capture 3D positioning and sacrifice quality and coherence. In this work, we introduce ComposeAnything, a novel framework for improving compositional image generation without retraining existing T2I models. Our approach first leverages the chain-of-thought reasoning abilities of LLMs to produce 2.5D semantic layouts from text, consisting of 2D object bounding boxes enriched with depth information and detailed captions. Based on this layout, we generate a spatial and depth aware coarse composite of objects that captures the intended composition, serving as a strong and interpretable prior that replaces stochastic noise initialization in diffusion-based T2I models. This prior guides the denoising process through object prior reinforcement and spatial-controlled denoising, enabling seamless generation of compositional objects and coherent backgrounds, while allowing refinement of inaccurate priors. ComposeAnything outperforms state-of-the-art methods on the T2I-CompBench and NSR-1K benchmarks for prompts with 2D/3D spatial arrangements, high object counts, and surreal compositions. Human evaluations further demonstrate that our model generates high-quality images with compositions that faithfully reflect the text.
Abstract (translated)
从文本生成包含复杂和新颖对象排列的图像，对于当前的文字到图像（T2I）模型来说仍是一个重大挑战。虽然以前基于布局的方法通过使用二维布局中的空间约束来改进对象排列，但它们常常难以捕捉三维定位，并且会牺牲质量和连贯性。在这项工作中，我们引入了ComposeAnything，这是一个新颖的框架，在不重新训练现有T2I模型的情况下，可以提高图像生成的组合能力。 我们的方法首先利用大语言模型（LLM）的链式思维推理能力，从文本中产生包含深度信息和详细说明的二维对象边界框的2.5D语义布局。基于这种布局，我们根据空间和深度感知生成一个粗略的复合图，该图捕获了预期的组合，并作为强大的、可解释的先验条件替代扩散型T2I模型中的随机噪声初始化。 这个先验通过物体优先级强化和空间控制去噪指导降噪过程，使复杂对象和连贯背景的无缝生成成为可能，同时允许对不准确的先验进行细化。ComposeAnything在具有二维/三维空间布局、高对象计数和超现实组合提示的T2I-CompBench和NSR-1K基准测试中超越了最先进的方法。 人类评估进一步表明，我们的模型能够生成高质量且忠实反映文本内容组成的图像。
URL
https://arxiv.org/abs/2505.24086
