Abstract
Recent advances in audio-driven talking head generation have achieved impressive results in lip synchronization and emotional expression. However, they largely overlook the crucial task of facial attribute editing. This capability is crucial for achieving deep personalization and expanding the range of practical applications, including user-tailored digital avatars, engaging online education content, and brand-specific digital customer service. In these key domains, the flexible adjustment of visual attributes-such as hairstyle, accessories, and subtle facial features is essential for aligning with user preferences, reflecting diverse brand identities, and adapting to varying contextual demands. In this paper, we present FaceEditTalker, a unified framework that enables controllable facial attribute manipulation while generating high-quality, audio-synchronized talking head videos. Our method consists of two key components: an image feature space editing module, which extracts semantic and detail features and allows flexible control over attributes like expression, hairstyle, and accessories; and an audio-driven video generation module, which fuses these edited features with audio-guided facial landmarks to drive a diffusion-based generator. This design ensures temporal coherence, visual fidelity, and identity preservation across frames. Extensive experiments on public datasets demonstrate that our method outperforms state-of-the-art approaches in lip-sync accuracy, video quality, and attribute controllability. Project page: this https URL
Abstract (translated)
近期在音频驱动的人脸生成领域的进展已经取得了唇部同步和情感表达方面的显著成果。然而，这些进步很大程度上忽略了关键的面部属性编辑任务。这一能力对于实现深度个性化以及扩大实际应用范围至关重要，包括用户定制的数字头像、互动性强的在线教育内容及品牌特定的数字化客户服务等。在这些重要领域中，能够灵活调整视觉属性（如发型、配饰和微妙的脸部特征）以符合用户的偏好、反映多样的品牌形象并适应不同的上下文需求是必不可少的。 在此论文中，我们提出了FaceEditTalker框架，该框架能够在生成高质量的音频同步人脸视频的同时实现可控制的面部属性编辑。我们的方法包含两个关键组件：一个图像特性空间编辑模块，它提取语义和细节特征，并允许对诸如表情、发型和配饰等属性进行灵活控制；以及一个基于音频驱动的视频生成模块，该模块将这些经过修改的特征与音频引导的人脸地标融合起来以驱动扩散型生成器。这种设计确保了帧间的时间一致性、视觉真实性和身份保持。 在公共数据集上的广泛实验表明，我们的方法在唇部同步准确性、视频质量和属性可控制性方面均优于当前最先进方法的表现。项目页面：[此链接](this https URL)
URL
https://arxiv.org/abs/2505.22141
