Abstract
One-shot styled handwriting image generation, despite achieving impressive results in recent years, remains challenging due to the difficulty in capturing the intricate and diverse characteristics of human handwriting by using solely a single reference image. Existing methods still struggle to generate visually appealing and realistic handwritten images and adapt to complex, unseen writer styles, struggling to isolate invariant style features (e.g., slant, stroke width, curvature) while ignoring irrelevant noise. To tackle this problem, we introduce Patch Contrastive Enhancement and Style-Aware Quantization via Denoising Diffusion (CONSTANT), a novel one-shot handwriting generation via diffusion model. CONSTANT leverages three key innovations: 1) a Style-Aware Quantization (SAQ) module that models style as discrete visual tokens capturing distinct concepts; 2) a contrastive objective to ensure these tokens are well-separated and meaningful in the embedding style space; 3) a latent patch-based contrastive (LLatentPCE) objective help improving quality and local structures by aligning multiscale spatial patches of generated and real features in latent space. Extensive experiments and analysis on benchmark datasets from multiple languages, including English, Chinese, and our proposed ViHTGen dataset for Vietnamese, demonstrate the superiority of adapting to new reference styles and producing highly detailed images of our method over state-of-the-art approaches. Code is available at GitHub
Abstract (translated)
近年来，一次性风格化手写图像生成虽然取得了令人瞩目的成果，但仍然面临着挑战。由于仅使用单一参考图像难以捕捉人类书写中细微而多样的特征，现有的方法在生成视觉上吸引人且逼真的手写图像以及适应复杂的、未见过的写作风格方面仍存在困难。它们难以分离不变的风格特征（如倾斜度、笔画宽度和曲线），同时忽略无关的噪声。 为了应对这一问题，我们提出了一种名为“Patch Contrastive Enhancement and Style-Aware Quantization via Denoising Diffusion”(CONSTANT)的新颖一次性手写生成方法，该方法基于扩散模型。CONSTANT采用三个关键创新： 1. **风格感知量化（SAQ）模块**：通过捕捉不同概念的离散视觉令牌来建模风格。 2. 对比性目标确保这些令牌在嵌入式风格空间中彼此分离且有意义。 3. 基于潜在补丁对比的目标 (LLatentPCE)，通过在潜空间中对齐生成和真实特征的多尺度空间补丁，提高图像质量和局部结构。 我们在多个语言的数据集上进行了广泛的实验和分析，包括英语、中文以及我们提出的越南语ViHTGen数据集。这些实验证明了我们的方法在适应新参考风格和产生高质量详细图像方面优于现有的最佳方法。代码可在GitHub上获取。
URL
https://arxiv.org/abs/2603.07543
