Abstract
Text-to-Image (T2I) models have recently achieved remarkable success in generating images from textual descriptions. However, challenges still persist in accurately rendering complex scenes where actions and interactions form the primary semantic focus. Our key observation in this work is that T2I models frequently struggle to capture nuanced and often implicit attributes inherent in action depiction, leading to generating images that lack key contextual details. To enable systematic evaluation, we introduce AcT2I, a benchmark designed to evaluate the performance of T2I models in generating images from action-centric prompts. We experimentally validate that leading T2I models do not fare well on AcT2I. We further hypothesize that this shortcoming arises from the incomplete representation of the inherent attributes and contextual dependencies in the training corpora of existing T2I models. We build upon this by developing a training-free, knowledge distillation technique utilizing Large Language Models to address this limitation. Specifically, we enhance prompts by incorporating dense information across three dimensions, observing that injecting prompts with temporal details significantly improves image generation accuracy, with our best model achieving an increase of 72%. Our findings highlight the limitations of current T2I methods in generating images that require complex reasoning and demonstrate that integrating linguistic knowledge in a systematic way can notably advance the generation of nuanced and contextually accurate images.
Abstract (translated)
最近，文本到图像（Text-to-Image，T2I）模型在从文字描述生成图片方面取得了显著的成功。然而，在渲染以动作和互动为主要语义焦点的复杂场景时，依然存在挑战。我们在这项工作中的关键观察是：T2I模型经常难以捕捉行动描绘中内在的细微且常常隐含的特点，导致生成的图像缺少重要的上下文细节。 为了进行系统的评估，我们引入了AcT2I这一基准测试平台，旨在评价T2I模型在处理以动作为中心提示时的表现。通过实验验证，我们发现领先的T2I模型在AcT2I上的表现不尽如人意。进一步地，我们假设这种不足源于现有T2I模型训练数据中内在属性和上下文依赖关系的不完整表示。 为了解决这一限制，我们开发了一种无需重新训练的知识蒸馏技术，利用大型语言模型进行改进。具体而言，通过在提示语中加入跨三个维度的密集信息，特别是增加时间细节，显著提高了图像生成的准确性。我们的最佳模型在此方法下实现了72%的性能提升。 这些发现揭示了目前T2I方法在生成需要复杂推理的图片时存在的局限性，并表明系统地整合语言知识可以明显提高生成细腻且上下文准确度高的图像的能力。
URL
https://arxiv.org/abs/2509.16141
