Abstract
Audio-driven talking head generation requires precise synchronization between facial animations and audio signals. This paper introduces ATL-Diff, a novel approach addressing synchronization limitations while reducing noise and computational costs. Our framework features three key components: a Landmark Generation Module converting audio to facial landmarks, a Landmarks-Guide Noise approach that decouples audio by distributing noise according to landmarks, and a 3D Identity Diffusion network preserving identity characteristics. Experiments on MEAD and CREMA-D datasets demonstrate that ATL-Diff outperforms state-of-the-art methods across all metrics. Our approach achieves near real-time processing with high-quality animations, computational efficiency, and exceptional preservation of facial nuances. This advancement offers promising applications for virtual assistants, education, medical communication, and digital platforms. The source code is available at: \href{this https URL}{this https URL}
Abstract (translated)
基于音频的面部动画生成需要面部动画与音频信号之间精确同步。本文介绍了一种名为ATL-Diff的新方法，该方法解决了同步限制问题，并在减少噪声和计算成本方面表现出色。我们的框架包含三个关键组件：一个将音频转换为面部标志的关键点生成模块、一个通过根据面部标志分布噪声来解耦音频的Landmarks-Guide噪声方法以及一个保留身份特征的3D Identity Diffusion网络。 我们在MEAD和CREMA-D数据集上的实验表明，ATL-Diff在所有评估指标上均优于现有最先进的方法。我们的方法实现了接近实时处理、高质量动画生成、计算效率，并且在面部细节保留方面表现出色。这项进步为虚拟助手、教育、医疗沟通和数字平台的应用提供了广阔前景。 源代码可以在以下网址获取：[此处提供具体的URL链接]
URL
https://arxiv.org/abs/2507.12804
