Abstract
Face reenactment aims to generate realistic talking head videos by transferring motion from a driving video to a static source image while preserving the source identity. Although existing methods based on either implicit or explicit keypoints have shown promise, they struggle with large pose variations due to warping artifacts or the limitations of coarse facial landmarks. In this paper, we present the Face Reenactment Video Diffusion model (FRVD), a novel framework for high-fidelity face reenactment under large pose changes. Our method first employs a motion extractor to extract implicit facial keypoints from the source and driving images to represent fine-grained motion and to perform motion alignment through a warping module. To address the degradation introduced by warping, we introduce a Warping Feature Mapper (WFM) that maps the warped source image into the motion-aware latent space of a pretrained image-to-video (I2V) model. This latent space encodes rich priors of facial dynamics learned from large-scale video data, enabling effective warping correction and enhancing temporal coherence. Extensive experiments show that FRVD achieves superior performance over existing methods in terms of pose accuracy, identity preservation, and visual quality, especially in challenging scenarios with extreme pose variations.
Abstract (translated)
面部重演的目标是通过将驱动视频中的运动转移到静态源图像上，生成逼真的说话头像视频，同时保持源身份的完整性。尽管基于隐式或显式关键点的方法已经显示出潜力，但它们在处理大姿态变化时由于扭曲伪影或粗略面部标志的局限性而面临挑战。 在这篇论文中，我们介绍了Face Reenactment Video Diffusion模型（FRVD），这是一种用于处理大幅度姿势变化下的高质量面部重演的新颖框架。我们的方法首先采用了一个运动提取器来从源图像和驱动图像中隐式地提取面部关键点，以表示精细的运动并通过一个扭曲模块进行运动对齐。为了解决由于扭曲引入的质量下降问题，我们提出了一种Warped Feature Mapper（WFM），它将扭曲后的源图像映射到预训练的图像到视频(I2V)模型的感知运动潜在空间中。该潜在空间包含了从大规模视频数据中学到的人脸动态先验知识，这使得有效的扭曲纠正成为可能，并增强了时间一致性。 广泛的实验表明，在姿态准确性、身份保持和视觉质量方面，FRVD在包括极端姿势变化在内的具有挑战性的场景下都超越了现有的方法。
URL
https://arxiv.org/abs/2507.16341
