Abstract
Existing handwritten text generation methods often require more than ten handwriting samples as style references. However, in practical applications, users tend to prefer a handwriting generation model that operates with just a single reference sample for its convenience and efficiency. This approach, known as "one-shot generation", significantly simplifies the process but poses a significant challenge due to the difficulty of accurately capturing a writer's style from a single sample, especially when extracting fine details from the characters' edges amidst sparse foreground and undesired background noise. To address this problem, we propose a One-shot Diffusion Mimicker (One-DM) to generate handwritten text that can mimic any calligraphic style with only one reference sample. Inspired by the fact that high-frequency information of the individual sample often contains distinct style patterns (e.g., character slant and letter joining), we develop a novel style-enhanced module to improve the style extraction by incorporating high-frequency components from a single sample. We then fuse the style features with the text content as a merged condition for guiding the diffusion model to produce high-quality handwritten text images. Extensive experiments demonstrate that our method can successfully generate handwriting scripts with just one sample reference in multiple languages, even outperforming previous methods using over ten samples. Our source code is available at this https URL.
Abstract (translated)
现有的手写文本生成方法通常需要超过10个手写样本作为样式参考。然而，在实际应用中，用户倾向于更喜欢只需一个手写样本操作的单样本生成模型，因为这样既方便又高效。这种方法被称为“一次生成”，显著简化了工艺流程，但由于准确捕捉一个写者的风格从单个样本的难度较大，尤其是在从字符边缘提取细小细节的过程中，尤其是在稀疏的前景和不需要的背景噪音中，这个问题仍然存在。为了应对这个问题，我们提出了一个一次生成扩散模仿者（One-DM）来生成具有任何楷体风格的只有一单个样本的手写文本。受到单个样本中高频信息通常包含不同的风格模式（例如，字符倾斜和字母连接）的影响，我们开发了一个新的样式增强模块，通过将高频组件从单个样本中集成来提高样式提取。然后将样式特征与文本内容作为合并条件，引导扩散模型产生高质量的手写文本图像。大量实验证明，我们的方法可以在多种语言中成功生成只需一个样本参考的手写文本，甚至超过了使用超过10个样本的前人方法。我们的源代码可在此处访问：https://url.cn/
URL
https://arxiv.org/abs/2409.04004
