Abstract
Despite advancements in text-to-image models, generating images that precisely align with textual descriptions remains challenging due to misalignment in training data. In this paper, we analyze the critical role of caption precision and recall in text-to-image model training. Our analysis of human-annotated captions shows that both precision and recall are important for text-image alignment, but precision has a more significant impact. Leveraging these insights, we utilize Large Vision Language Models to generate synthetic captions for training. Models trained with these synthetic captions show similar behavior to those trained on human-annotated captions, underscores the potential for synthetic data in text-to-image training.
Abstract (translated)
尽管文本到图像模型取得了进步，生成与文本描述精确匹配的图像仍然具有挑战性，主要是因为训练数据中的对齐问题。在本文中，我们分析了标题（caption）的精度和召回率在文本到图像模型训练中的关键作用。通过对人工标注的标题进行分析，我们发现对于文本-图像的对齐而言，精度和召回率都很重要，但精度的影响更大。基于这些见解，我们利用大型视觉语言模型生成合成标题用于训练。使用这些合成标题训练出的模型表现出与使用人工标注标题训练的模型类似的行为，这突显了合成数据在文本到图像训练中的潜在价值。
URL
https://arxiv.org/abs/2411.05079
