Abstract
Text-to-image (T2I) generation model has made significant advancements, resulting in high-quality images aligned with an input prompt. However, despite T2I generation's ability to generate fine-grained images, it still faces challenges in accurately generating images when the input prompt contains complex concepts, especially human pose. In this paper, we propose PointT2I, a framework that effectively generates images that accurately correspond to the human pose described in the prompt by using a large language model (LLM). PointT2I consists of three components: Keypoint generation, Image generation, and Feedback system. The keypoint generation uses an LLM to directly generate keypoints corresponding to a human pose, solely based on the input prompt, without external references. Subsequently, the image generation produces images based on both the text prompt and the generated keypoints to accurately reflect the target pose. To refine the outputs of the preceding stages, we incorporate an LLM-based feedback system that assesses the semantic consistency between the generated contents and the given prompts. Our framework is the first approach to leveraging LLM for keypoints-guided image generation without any fine-tuning, producing accurate pose-aligned images based solely on textual prompts.
Abstract (translated)
文本到图像（T2I）生成模型已经取得了显著进展，能够根据输入提示生成高质量的图像。然而，尽管T2I生成模型能够在某些方面产生细节丰富的图像，当输入提示包含复杂的概念时，尤其是人体姿态时，仍然面临准确生成图像的挑战。 在本文中，我们提出了PointT2I框架，该框架利用大规模语言模型（LLM）有效地生成与提示中描述的人体姿态相匹配的高质量图像。PointT2I包括三个组成部分：关键点生成、图像生成和反馈系统。 1. **关键点生成**：这部分使用一个大模型直接根据输入提示生成对应人体姿势的关键点，而不依赖于外部参考。 2. **图像生成**：该部分基于文本提示以及由前面步骤产生的关键点来生成图像，以准确反映目标姿态。 3. **反馈系统**：为了优化前两个阶段的输出，我们引入了一个基于大模型的反馈机制，它评估生成内容与给定提示之间的语义一致性。 我们的框架是首个不进行微调即可利用LLM引导关键点并据此生成精确对齐的人体姿势图像的方法。仅通过文本提示，PointT2I就能产生准确的姿态对齐图像。
URL
https://arxiv.org/abs/2506.01370
