Abstract
Previous deepfake detection methods mostly depend on low-level textural features vulnerable to perturbations and fall short of detecting unseen forgery methods. In contrast, high-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. Motivated by this, we propose a detection method that utilizes high-level semantic features of faces to identify inconsistencies in temporal domain. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video classification network, initialized with a meta-functional face encoder for enriched facial representation. In this way, we can take advantage of both the powerful spatio-temporal model and the high-level semantic information of faces. Furthermore, to leverage easily accessible real face data and guide the model in focusing on spatio-temporal features, we design a Dynamic Video Self-Blending (DVSB) method to efficiently generate training samples with diverse spatio-temporal forgery traces using real facial videos. Based on this, we advance our framework with a two-stage training approach: The first stage employs a novel self-supervised contrastive learning, where we encourage the network to focus on forgery traces by impelling videos generated by the same forgery process to have similar representations. On the basis of the representation learned in the first stage, the second stage involves fine-tuning on face forgery detection dataset to build a deepfake detector. Extensive experiments validates that UniForensics outperforms existing face forgery methods in generalization ability and robustness. In particular, our method achieves 95.3\% and 77.2\% cross dataset AUC on the challenging Celeb-DFv2 and DFDC respectively.
Abstract (translated)
先前的深度伪造检测方法主要依赖于低水平的纹理特征，这些特征易受扰动，并且无法检测出未被标识为伪造的方法。相反，高层面的语义特征对扰动更敏感，不受伪造特定瑕疵的限制，因此具有更强的一般化能力。基于这一点，我们提出了利用人脸面部高层语义特征来识别时域中存在差异的检测方法。我们引入了UniForensics，这是一种新颖的深度伪造检测框架，利用Transformer类视频分类网络作为启动器，该网络初始化了一个元功能脸部编码器，以丰富面部表示。通过这种方式，我们可以充分利用强大的时空模型和面部高级语义信息的优势。此外，为了利用易于访问的真实人脸数据，并引导模型集中注意力于时空特征，我们设计了一个动态视频自融合（DVSB）方法，有效地使用真实的人脸视频生成多样化的时空伪造痕迹训练样本，从而在第二阶段引入了深度学习框架的改进：首先，我们提出了一个新颖的监督对抗性学习策略，通过促使从同一伪造过程中产生的视频具有相似表示，来引导网络关注伪造足迹。基于第一阶段所学到的代表，在第二阶段中涉及对人脸伪造检测数据集进行微调以构建深度伪造检测器。广泛的实验验证了UniForensics在一般化能力和鲁棒性方面优于现有的面部伪造方法。具体来说，我们的方法在挑战性的Celeb-DFv2和DFDC数据集中分别实现95.3\%和77.2\%的交叉数据AUC。 总结：利用高层面的语义特征检测深度伪造是一个潜在的研究领域，并且可以增强现有的框架能力。我们提出了一个综合解决方案，结合了自监督强化学习策略与分层训练方法，以改进深度伪造检测器的表现。
URL
https://arxiv.org/abs/2407.19079
