Abstract
Deepfake techniques generate highly realistic data, making it challenging for humans to discern between actual and artificially generated images. Recent advancements in deep learning-based deepfake detection methods, particularly with diffusion models, have shown remarkable progress. However, there is a growing demand for real-world applications to detect unseen individuals, deepfake techniques, and scenarios. To address this limitation, we propose a Prototype-based Unified Framework for Deepfake Detection (PUDD). PUDD offers a detection system based on similarity, comparing input data against known prototypes for video classification and identifying potential deepfakes or previously unseen classes by analyzing drops in similarity. Our extensive experiments reveal three key findings: (1) PUDD achieves an accuracy of 95.1% on Celeb-DF, outperforming state-of-the-art deepfake detection methods; (2) PUDD leverages image classification as the upstream task during training, demonstrating promising performance in both image classification and deepfake detection tasks during inference; (3) PUDD requires only 2.7 seconds for retraining on new data and emits 10$^{5}$ times less carbon compared to the state-of-the-art model, making it significantly more environmentally friendly.
Abstract (translated)
深度伪造技术生成的数据非常逼真，使得人类很难区分真实和伪造的图像。在深度学习为基础的深度伪造检测方法的最新进展中，特别是扩散模型，取得了显著的进步。然而，对于检测未见过的个体、深度伪造技术和情景，现实世界的应用需求不断增加。为了克服这一局限，我们提出了一个基于原型统一的深度伪造检测框架（PUDD）。PUDD 提供了一种基于相似度的检测系统，将输入数据与已知原型进行比较，通过分析相似性的下降来识别潜在的深度伪造者或以前未见过的类别。我们进行的广泛实验揭示了三个关键发现：（1）PUDD 在Celeb-DF上的准确率达到了95.1%，超越了最先进的深度伪造检测方法；（2）在训练过程中将图像分类作为上游任务，表明在推理过程中在图像分类和深度伪造检测任务上都取得了有前景的表现；（3）在只使用新的数据进行重新训练的情况下，PUDD 只需要2.7秒钟，排放的碳比最先进模型少10^5倍，使得它具有显著的环保优势。
URL
https://arxiv.org/abs/2406.15921
