Abstract
In this paper, we propose TextDestroyer, the first training- and annotation-free method for scene text destruction using a pre-trained diffusion model. Existing scene text removal models require complex annotation and retraining, and may leave faint yet recognizable text information, compromising privacy protection and content concealment. TextDestroyer addresses these issues by employing a three-stage hierarchical process to obtain accurate text masks. Our method scrambles text areas in the latent start code using a Gaussian distribution before reconstruction. During the diffusion denoising process, self-attention key and value are referenced from the original latent to restore the compromised background. Latent codes saved at each inversion step are used for replacement during reconstruction, ensuring perfect background restoration. The advantages of TextDestroyer include: (1) it eliminates labor-intensive data annotation and resource-intensive training; (2) it achieves more thorough text destruction, preventing recognizable traces; and (3) it demonstrates better generalization capabilities, performing well on both real-world scenes and generated images.
Abstract (translated)
在这篇论文中，我们提出了TextDestroyer，这是一种无需训练和标注的场景文本破坏方法，利用预训练扩散模型实现。现有的场景文本去除模型需要复杂的标注和重新训练，并且可能会留下模糊但可识别的文字信息，从而影响隐私保护和内容隐藏。TextDestroyer通过采用三阶段分层过程来获取准确的文本掩码，解决了这些问题。我们的方法在重建之前使用高斯分布对潜在代码中的文本区域进行扰乱。在扩散去噪过程中，引用原始潜变量的自注意力键和值以恢复受损背景。每个逆向步骤保存的潜在编码用于重建时替换，确保完美恢复背景。TextDestroyer的优势包括：(1) 消除了劳动密集型数据标注和资源密集型训练；(2) 实现了更彻底的文字破坏，防止可识别痕迹的存在；(3) 展示了更好的泛化能力，在真实场景图像和生成的图像上均表现良好。
URL
https://arxiv.org/abs/2411.00355
