Abstract
Stable Diffusion has advanced text-to-image synthesis, but training models to generate images with accurate object quantity is still difficult due to the high computational cost and the challenge of teaching models the abstract concept of quantity. In this paper, we propose CountDiffusion, a training-free framework aiming at generating images with correct object quantity from textual descriptions. CountDiffusion consists of two stages. In the first stage, an intermediate denoising result is generated by the diffusion model to predict the final synthesized image with one-step denoising, and a counting model is used to count the number of objects in this image. In the second stage, a correction module is used to correct the object quantity by changing the attention map of the object with universal guidance. The proposed CountDiffusion can be plugged into any diffusion-based text-to-image (T2I) generation models without further training. Experiment results demonstrate the superiority of our proposed CountDiffusion, which improves the accurate object quantity generation ability of T2I models by a large margin.
Abstract (translated)
稳定扩散模型在文本到图像的合成方面取得了进展，但训练模型生成具有准确对象数量的图像仍然困难，这主要是由于计算成本高昂以及向模型传授抽象的数量概念极具挑战性。本文提出了一种名为CountDiffusion的新框架，该框架旨在从文字描述中生成具有正确对象数量的图像，并且无需额外训练。CountDiffusion由两个阶段组成：第一阶段使用扩散模型通过一步去噪预测最终合成的图像并生成中间去噪结果，同时利用一个计数模型来计算此图像中的物体数量；第二阶段则采用修正模块根据通用引导改变物体的关注图以纠正对象的数量。 所提出的CountDiffusion可以无缝集成到任何基于扩散的文本到图像(T2I)生成模型中而无需进一步训练。实验结果显示了我们提出的方法CountDiffusion的优势，它大大提高了T2I模型准确生成对象数量的能力。
URL
https://arxiv.org/abs/2505.04347
