Abstract
Fine-grained text to image synthesis involves generating images from texts that belong to different categories. In contrast to general text to image synthesis, in fine-grained synthesis there is high similarity between images of different subclasses, and there may be linguistic discrepancy among texts describing the same image. Recent Generative Adversarial Networks (GAN), such as the Recurrent Affine Transformation (RAT) GAN model, are able to synthesize clear and realistic images from texts. However, GAN models ignore fine-grained level information. In this paper we propose an approach that incorporates an auxiliary classifier in the discriminator and a contrastive learning method to improve the accuracy of fine-grained details in images synthesized by RAT GAN. The auxiliary classifier helps the discriminator classify the class of images, and helps the generator synthesize more accurate fine-grained images. The contrastive learning method minimizes the similarity between images from different subclasses and maximizes the similarity between images from the same subclass. We evaluate on several state-of-the-art methods on the commonly used CUB-200-2011 bird dataset and Oxford-102 flower dataset, and demonstrated superior performance.
Abstract (translated)
细粒度文本到图像合成涉及从属于不同类别的文本生成图像。与一般的文本到图像合成相比，细粒度合成中不同子类的图像之间存在高度相似性，并且描述同一图像的文本可能存在语言上的差异。近期的生成对抗网络（GAN），如递归仿射变换（RAT）GAN模型，能够从文本生成清晰真实的图像。然而，GAN模型忽略了细粒度级别的信息。本文提出了一种方法，在判别器中加入了辅助分类器，并采用对比学习方法来提高由RAT GAN合成的图像中的细粒度细节准确性。辅助分类器帮助判别器对图像类别进行分类，并有助于生成器合成更准确的细粒度图像。对比学习方法最小化来自不同子类别的图像之间的相似性，同时最大化同一子类别的图像间的相似性。我们在常用的CUB-200-2011鸟类数据集和Oxford-102花卉数据集上评估了多种最先进的方法，并展示了优越的性能。
URL
https://arxiv.org/abs/2412.07196
