Abstract
We present Follow-Your-Emoji-Faster, an efficient diffusion-based framework for freestyle portrait animation driven by facial landmarks. The main challenges in this task are preserving the identity of the reference portrait, accurately transferring target expressions, and maintaining long-term temporal consistency while ensuring generation efficiency. To address identity preservation and accurate expression retargeting, we enhance Stable Diffusion with two key components: a expression-aware landmarks as explicit motion signals, which improve motion alignment, support exaggerated expressions, and reduce identity leakage; and a fine-grained facial loss that leverages both expression and facial masks to better capture subtle expressions and faithfully preserve the reference appearance. With these components, our model supports controllable and expressive animation across diverse portrait types, including real faces, cartoons, sculptures, and animals. However, diffusion-based frameworks typically struggle to efficiently generate long-term stable animation results, which remains a core challenge in this task. To address this, we propose a progressive generation strategy for stable long-term animation, and introduce a Taylor-interpolated cache, achieving a 2.6X lossless acceleration. These two strategies ensure that our method produces high-quality results efficiently, making it user-friendly and accessible. Finally, we introduce EmojiBench++, a more comprehensive benchmark comprising diverse portraits, driving videos, and landmark sequences. Extensive evaluations on EmojiBench++ demonstrate that Follow-Your-Emoji-Faster achieves superior performance in both animation quality and controllability. The code, training dataset and benchmark will be found in this https URL.
Abstract (translated)
我们提出了Follow-Your-Emoji-Faster，这是一种基于高效的扩散框架的自由风格肖像动画方法，该方法通过面部标志进行驱动。此任务的主要挑战在于保留参考肖像的身份、准确传递目标表情以及在确保生成效率的同时保持长期的时间一致性。为了解决身份保留和准确的表情重定向问题，我们通过两种关键组件增强了Stable Diffusion：一种是情感感知的面部标记作为明确的动作信号，这可以提高动作对齐能力、支持夸张的表情并且减少身份泄露；另一种是精细的人脸损失函数，该函数利用表情和面部遮罩来更好地捕捉细微的表情变化并忠实保留参考外观。通过这些组件的支持，我们的模型能够实现不同种类肖像（包括真实人脸、卡通人物、雕塑以及动物）的可控且富有表现力的动画效果。 然而，基于扩散框架的方法通常难以高效地生成长期稳定的动画结果，这是此任务中的核心挑战之一。为了解决这个问题，我们提出了一种渐进式生成策略以实现稳定长期的动画，并引入了泰勒插值缓存机制，实现了2.6倍无损加速的效果。这两种策略确保我们的方法能够高效生产高质量的结果，使其易于使用且普及性高。 最后，我们介绍了EmojiBench++，这是一个更全面的基准测试集合，包含多种肖像、驱动视频和面部标志序列。在EmojiBench++上的广泛评估表明，Follow-Your-Emoji-Faster在动画质量和可控性方面表现出色。相关代码、训练数据集以及基准测试可在[此链接](https://this.url.com)中找到。
URL
https://arxiv.org/abs/2509.16630
