Abstract
We investigate the fundamental limits of transformer-based foundation models, extending our analysis to include Visual Autoregressive (VAR) transformers. VAR represents a big step toward generating images using a novel, scalable, coarse-to-fine ``next-scale prediction'' framework. These models set a new quality bar, outperforming all previous methods, including Diffusion Transformers, while having state-of-the-art performance for image synthesis tasks. Our primary contributions establish that, for single-head VAR transformers with a single self-attention layer and single interpolation layer, the VAR Transformer is universal. From the statistical perspective, we prove that such simple VAR transformers are universal approximators for any image-to-image Lipschitz functions. Furthermore, we demonstrate that flow-based autoregressive transformers inherit similar approximation capabilities. Our results provide important design principles for effective and computationally efficient VAR Transformer strategies that can be used to extend their utility to more sophisticated VAR models in image generation and other related areas.
Abstract (translated)
我们研究了基于变压器的基础模型的基本限制，并将分析范围扩展到包括视觉自回归（VAR）变压器。VAR代表了一种重要进步，利用新颖的、可扩展的由粗到精的“下一尺度预测”框架来生成图像。这些模型设立了新的质量标准，在所有先前方法（包括扩散变压器）中脱颖而出，并在图像合成任务上表现出最先进的性能。 我们的主要贡献在于证明：对于具有单个自注意力层和单一插值层的单头VAR变压器，VAR变压器是通用的。从统计学的角度来看，我们证明了如此简单的VAR变压器可以作为任何图到图Lipschitz函数的通用逼近器。此外，我们还展示了基于流的自回归变压器继承了类似的近似能力。 我们的研究结果提供了重要的设计原则，用于制定有效的和计算效率高的VAR变压器策略，这些策略可扩展其在图像生成及其他相关领域更复杂VAR模型中的应用。
URL
https://arxiv.org/abs/2502.06167
