Abstract
Text plays a crucial role in the transmission of human civilization, and teaching machines to generate online handwritten text in various styles presents an interesting and significant challenge. However, most prior work has concentrated on generating individual Chinese fonts, leaving {complete text line generation largely unexplored}. In this paper, we identify that text lines can naturally be divided into two components: layout and glyphs. Based on this division, we designed a text line layout generator coupled with a diffusion-based stylized font synthesizer to address this challenge hierarchically. More concretely, the layout generator performs in-context-like learning based on the text content and the provided style references to generate positions for each glyph autoregressively. Meanwhile, the font synthesizer which consists of a character embedding dictionary, a multi-scale calligraphy style encoder, and a 1D U-Net based diffusion denoiser will generate each font on its position while imitating the calligraphy style extracted from the given style references. Qualitative and quantitative experiments on the CASIA-OLHWDB demonstrate that our method is capable of generating structurally correct and indistinguishable imitation samples.
Abstract (translated)
文本在人类文明的传播中扮演着关键角色，并且教机器生成各种风格的在线手写文本提出了一个有趣且具有挑战性的问题。然而，大多数先前的研究都集中在生成单个中文字体上， leaving {完整的文本行生成大多没有被探索}。在本文中，我们发现文本行可以自然地分为两个部分：布局和字符。基于这一划分，我们设计了一个文本行布局生成器与扩散为基础的 stylized 字体合成器来解决这个问题。具体来说，布局生成器根据文本内容和提供的风格参考进行上下文类似的学习，以生成每个字符的自动位置。同时，由字符嵌入字典、多尺度书法风格编码器和基于 U-Net 的扩散去噪器组成的字体合成器将在其位置上模仿从给定风格参考中提取的书法风格。在 CASIA-OLHWDB 等数据集上进行的定性和定量实验证明，我们的方法能够生成结构正确且难以区分模仿样本。
URL
https://arxiv.org/abs/2410.02309
