Abstract
Diffusion-based scene text synthesis has progressed rapidly, yet existing methods commonly rely on additional visual conditioning modules and require large-scale annotated data to support multilingual generation. In this work, we revisit the necessity of complex auxiliary modules and further explore an approach that simultaneously ensures glyph accuracy and achieves high-fidelity scene integration, by leveraging diffusion models' inherent capabilities for contextual reasoning. To this end, we introduce TextFlux, a DiT-based framework that enables multilingual scene text synthesis. The advantages of TextFlux can be summarized as follows: (1) OCR-free model architecture. TextFlux eliminates the need for OCR encoders (additional visual conditioning modules) that are specifically used to extract visual text-related features. (2) Strong multilingual scalability. TextFlux is effective in low-resource multilingual settings, and achieves strong performance in newly added languages with fewer than 1,000 samples. (3) Streamlined training setup. TextFlux is trained with only 1% of the training data required by competing methods. (4) Controllable multi-line text generation. TextFlux offers flexible multi-line synthesis with precise line-level control, outperforming methods restricted to single-line or rigid layouts. Extensive experiments and visualizations demonstrate that TextFlux outperforms previous methods in both qualitative and quantitative evaluations.
Abstract (translated)
基于扩散模型的场景文本合成技术取得了快速进展，但现有方法通常依赖额外的视觉条件模块，并且需要大规模标注数据来支持多语言生成。在这项工作中，我们重新审视了复杂辅助模块的必要性，并进一步探索了一种同时确保字形准确性并实现高质量场景整合的方法，通过利用扩散模型在上下文推理方面的内在能力。为此，我们介绍了基于DiT框架的TextFlux系统，它能够进行多语言场景文本合成。TextFlux的优势可以总结如下： 1. **无OCR架构**：TextFlux消除了专门用于提取视觉文本相关特征的OCR编码器（额外的视觉条件模块）的需求。 2. **强大的多语言可扩展性**：在低资源环境下，TextFlux仍然非常有效，并且在新添加的语言中使用少于1,000个样本也能实现强劲性能。 3. **简化的训练设置**：与竞争方法相比，TextFlux仅需1%的训练数据即可完成训练。 4. **可控多行文本生成**：TextFlux提供灵活的多行合成，并具有精确到每行级别的控制能力，超越了限制于单行或固定布局的方法。 大量的实验和可视化结果证明，在定性和定量评估中，TextFlux均优于先前的方法。
URL
https://arxiv.org/abs/2505.17778
