Abstract
The imitation of cursive handwriting is mainly limited to generating handwritten words or lines. Multiple synthetic outputs must be stitched together to create paragraphs or whole pages, whereby consistency and layout information are lost. To close this gap, we propose a method for imitating handwriting at the paragraph level that also works for unseen writing styles. Therefore, we introduce a modified latent diffusion model that enriches the encoder-decoder mechanism with specialized loss functions that explicitly preserve the style and content. We enhance the attention mechanism of the diffusion model with adaptive 2D positional encoding and the conditioning mechanism to work with two modalities simultaneously: a style image and the target text. This significantly improves the realism of the generated handwriting. Our approach sets a new benchmark in our comprehensive evaluation. It outperforms all existing imitation methods at both line and paragraph levels, considering combined style and content preservation.
Abstract (translated)
模仿草书手迹的主要限制在于只能生成手写单词或行。多个合成输出必须拼接在一起才能创建段落或整个页面，从而丢失了 consistency 和 layout 信息。为了填补这个空白，我们提出了一个模仿草书手迹的方法，该方法对于未见过的书写风格也有效。因此，我们引入了一个修改的潜在扩散模型，它通过专门损失函数丰富编码器-解码器机制，并保留风格和内容。我们通过自适应 2D 位置编码和条件机制来增强扩散模型的注意机制，以同时处理样式图像和目标文本。这使得生成的手写内容更真实。在我们的全面评估中，我们的方法为一个新的基准。它超越了所有现有的模仿方法，既考虑了风格，也考虑了内容保留。
URL
https://arxiv.org/abs/2409.00786
