Abstract
Face Forgery videos have elicited critical social public concerns and various detectors have been proposed. However, fully-supervised detectors may lead to easily overfitting to specific forgery methods or videos, and existing self-supervised detectors are strict on auxiliary tasks, such as requiring audio or multi-modalities, leading to limited generalization and robustness. In this paper, we examine whether we can address this issue by leveraging visual-only real face videos. To this end, we propose to learn the Natural Consistency representation (NACO) of real face videos in a self-supervised manner, which is inspired by the observation that fake videos struggle to maintain the natural spatiotemporal consistency even under unknown forgery methods and different perturbations. Our NACO first extracts spatial features of each frame by CNNs then integrates them into Transformer to learn the long-range spatiotemporal representation, leveraging the advantages of CNNs and Transformer on local spatial receptive field and long-term memory respectively. Furthermore, a Spatial Predictive Module~(SPM) and a Temporal Contrastive Module~(TCM) are introduced to enhance the natural consistency representation learning. The SPM aims to predict random masked spatial features from spatiotemporal representation, and the TCM regularizes the latent distance of spatiotemporal representation by shuffling the natural order to disturb the consistency, which could both force our NACO more sensitive to the natural spatiotemporal consistency. After the representation learning stage, a MLP head is fine-tuned to perform the usual forgery video classification task. Extensive experiments show that our method outperforms other state-of-the-art competitors with impressive generalization and robustness.
Abstract (translated)
面对伪造视频，已经引起了广泛的社会公众关注，并提出了各种检测方法。然而，完全监督的检测可能会导致对特定伪造方法的过拟合，而现有的自监督检测方法对辅助任务有严格的要求，导致对泛化和鲁棒性的限制。在本文中，我们探讨是否可以通过利用仅包含视觉信息的真实人脸视频来解决这个问题。为此，我们提出了一种自监督地学习真实人脸视频的自然一致性表示（NACO）的方法，这是基于观察到假视频在未知伪造方法下甚至无法保持自然时空一致性的启示。我们的NACO首先通过CNN提取每个帧的时空特征，然后将它们整合到Transformer中学习长距离时空表示，利用CNN和Transformer在局部时空感受野和长期记忆方面的优势。此外，还引入了空间预测模块（SPM）和时间对比模块（TCM）来增强自然一致性表示的学习。SPM旨在预测从时空表示中随机遮罩的时空特征，而TCM通过随机打乱自然顺序来干扰一致性，这可能会使我们的人脸NACO更加敏感。在表示学习阶段之后，对MLP头进行微调以执行通常的伪造视频分类任务。大量实验证明，我们的方法在泛化能力和鲁棒性方面优于其他最先进的竞争对手。
URL
https://arxiv.org/abs/2407.10550
