Abstract
The task of scene text editing is to modify or add texts on images while maintaining the fidelity of newly generated text and visual coherence with the background. Recent works based on latent diffusion models (LDM) show improved text editing results, yet still face challenges and often generate inaccurate or unrecognizable characters, especially for non-Latin ones (\eg, Chinese), which have complex glyph structures. To address these issues, we present FLUX-Text, a simple and advanced multilingual scene text editing framework based on FLUX-Fill. Specifically, we carefully investigate glyph conditioning, considering both visual and textual modalities. To retain the original generative capabilities of FLUX-Fill while enhancing its understanding and generation of glyphs, we propose lightweight glyph and text embedding modules. Owning to the lightweight design, FLUX-Text is trained only with $100K$ training examples compared to current popular methods trained with 2.9M ones. With no bells and whistles, our method achieves state-of-the-art performance on text editing tasks. Qualitative and quantitative experiments on the public datasets demonstrate that our method surpasses previous works in text fidelity.
Abstract (translated)
场景文本编辑的任务是在保持新生成的文字质量和与背景图像视觉一致性的前提下，修改或增加图片中的文字。最近基于潜在扩散模型（LDM）的工作提高了文本编辑的效果，但仍然面临挑战，并且常常产生不准确或难以辨认的字符，尤其是对于非拉丁文（如中文），这些文字具有复杂的字形结构。为了应对这些问题，我们提出了FLUX-Text，这是一个基于FLUX-Fill的简单而先进的多语言场景文本编辑框架。具体来说，我们仔细研究了字形条件化，同时考虑视觉和文本模态。 为了保留FLUX-Fill原有的生成能力并增强其对字形的理解与生成，我们提出了一种轻量级的字形和文本嵌入模块。由于采用了轻量化设计，FLUX-Text仅使用10万个训练样本进行训练，而目前流行的方法则需要290万个训练样本。在没有任何复杂技巧的情况下，我们的方法在文本编辑任务上达到了最先进的性能水平。通过公共数据集上的定性和定量实验表明，我们的方法在文字保真度方面超越了之前的工作。
URL
https://arxiv.org/abs/2505.03329
