Abstract
Autoregressive conditional image generation algorithms are capable of generating photorealistic images that are consistent with given textual or image conditions, and have great potential for a wide range of applications. Nevertheless, the majority of popular autoregressive image generation methods rely heavily on vector quantization, and the inherent discrete characteristic of codebook presents a considerable challenge to achieving high-quality image generation. To address this limitation, this paper introduces a novel conditional introduction network for continuous masked autoregressive models. The proposed self-control network serves to mitigate the negative impact of vector quantization on the quality of the generated images, while simultaneously enhancing the conditional control during the generation process. In particular, the self-control network is constructed upon a continuous mask autoregressive generative model, which incorporates multimodal conditional information, including text and images, into a unified autoregressive sequence in a serial manner. Through a self-attention mechanism, the network is capable of generating images that are controllable based on specific conditions. The self-control network discards the conventional cross-attention-based conditional fusion mechanism and effectively unifies the conditional and generative information within the same space, thereby facilitating more seamless learning and fusion of multimodal features.
Abstract (translated)
自回归条件图像生成算法能够根据给定的文本或图像条件生成逼真的照片，并且在广泛的应用中具有巨大的潜力。然而，大多数流行的自回归图像生成方法严重依赖于向量量化，而代码簿固有的离散特性对实现高质量图像生成构成了相当大的挑战。为了解决这一局限性，本文介绍了一种用于连续掩码自回归模型的新颖条件引入网络。所提出的自我控制网络旨在减轻向量量化对生成图像质量的负面影响，并同时增强生成过程中的条件控制能力。特别地，自我控制网络基于连续掩码自回归生成模型构建，该模型以序列方式将包括文本和图像在内的多模态条件信息融合到统一的自回归序列中。通过自我注意力机制，网络能够根据特定条件生成可控的图像。自我控制网络摒弃了传统的基于交叉注意力的条件融合机制，并有效地在同一空间内统一了条件信息与生成信息，从而促进了多模态特征学习和融合的更无缝衔接。
URL
https://arxiv.org/abs/2412.13635
