Abstract
Although chain-of-thought reasoning and reinforcement learning (RL) have driven breakthroughs in NLP, their integration into generative vision models remains underexplored. We introduce ReasonGen-R1, a two-stage framework that first imbues an autoregressive image generator with explicit text-based "thinking" skills via supervised fine-tuning on a newly generated reasoning dataset of written rationales, and then refines its outputs using Group Relative Policy Optimization. To enable the model to reason through text before generating images, We automatically generate and release a corpus of model crafted rationales paired with visual prompts, enabling controlled planning of object layouts, styles, and scene compositions. Our GRPO algorithm uses reward signals from a pretrained vision language model to assess overall visual quality, optimizing the policy in each update. Evaluations on GenEval, DPG, and the T2I benchmark demonstrate that ReasonGen-R1 consistently outperforms strong baselines and prior state-of-the-art models. More: this http URL.
Abstract (translated)
尽管基于链式思维推理和强化学习（RL）在自然语言处理（NLP）领域取得了突破性进展，但它们在生成视觉模型中的整合仍处于探索阶段。我们引入了ReasonGen-R1框架，这是一个两阶段的架构：首先通过在新生成的推理数据集上进行监督微调，赋予自回归图像生成器显式的基于文本的“思考”技能；然后利用群体相对策略优化（GRPO）来改进其输出结果。为了使模型能够在生成图像之前通过文本进行推理，我们自动创建并发布了一套由模型创作的论述文集，这些文档与视觉提示配对，从而能够控制物体布局、风格和场景构图的设计。我们的GRPO算法使用预训练的视觉语言模型提供的奖励信号来评估整体视觉质量，并在每次更新中优化策略。在GenEval、DPG以及T2I基准测试上的评估结果表明，ReasonGen-R1始终优于强大的基线模型及先前最先进的模型。更多信息，请参见：[此链接](http://this.http.url)。 注意：原文中的“this http URL”是一个占位符或伪链接，并非实际链接地址，在正式发布时应替换为有效的链接地址。
URL
https://arxiv.org/abs/2505.24875
