Abstract
Unified multimodal models integrate the reasoning capacity of large language models with both image understanding and generation, showing great promise for advanced multimodal intelligence. However, the community still lacks a rigorous reasoning-centric benchmark to systematically evaluate the alignment between understanding and generation, and their generalization potential in complex visual tasks. To this end, we introduce \textbf{GIR-Bench}, a comprehensive benchmark that evaluates unified models across three complementary perspectives. Firstly, we investigate understanding-generation consistency (GIR-Bench-UGC), asking whether models can consistently leverage the same knowledge in both understanding and generation tasks. Secondly, we investigate whether models can perform reasoning-centric text-to-image generation that requires applying logical constraints and implicit knowledge to generate faithful visual content (GIR-Bench-T2I). Thirdly, we evaluate whether models can handle multi-step reasoning in editing (GIR-Bench-Edit). For each subset, we carefully design different task-specific evaluation pipelines tailored for each task. This enables fine-grained and interpretable evaluation while mitigating biases from the prevalent MLLM-as-a-Judge paradigm. Extensive ablations over various unified models and generation-only systems have shown that: Although unified models are more capable of reasoning-driven visual tasks, they still exhibit a persistent gap between understanding and generation. The data and code for GIR-Bench are available at \href{this https URL}{this https URL}.
Abstract (translated)
统一的多模态模型将大型语言模型的推理能力与图像理解和生成相结合，展示了先进多模态智能的巨大潜力。然而，社区仍然缺乏一个以推理为中心的严格基准来系统地评估理解与生成之间的对齐情况以及它们在复杂视觉任务中的泛化潜力。为此，我们引入了**GIR-Bench**这一全面基准，它从三个方面综合评价统一模型：首先，我们探讨理解和生成的一致性（GIR-Bench-UGC），即模型是否能在理解和生成任务中一致地利用相同的知识；其次，我们研究模型能否执行以推理为中心的文本到图像生成，这需要应用逻辑约束和隐含知识来生成忠实的视觉内容（GIR-Bench-T2I）；最后，我们评估模型处理编辑中的多步推理的能力（GIR-Bench-Edit）。对于每个子集，我们都精心设计了不同的任务特定评估流程。这一方法使得评价更加细致入微且具有可解释性，并能减轻MLLM-as-a-Judge范式带来的偏见。 通过对各种统一模型和仅生成系统的广泛消融研究发现：尽管统一模型在推理驱动的视觉任务上更擅长，它们在理解和生成之间仍然存在持续性的差距。GIR-Bench的数据和代码可在[此处](this https URL)获取。
URL
https://arxiv.org/abs/2510.11026
