Abstract
Multimedia data, particularly images and videos, is integral to various applications, including surveillance, visual interaction, biometrics, evidence gathering, and advertising. However, amateur or skilled counterfeiters can simulate them to create deepfakes, often for slanderous motives. To address this challenge, several forensic methods have been developed to ensure the authenticity of the content. The effectiveness of these methods depends on their focus, with challenges arising from the diverse nature of manipulations. In this article, we analyze existing forensic methods and observe that each method has unique strengths in detecting deepfake traces by focusing on specific facial regions, such as the frame, face, lips, eyes, or nose. Considering these insights, we propose a novel hybrid approach called Face2Parts based on hierarchical feature representation ($HFR$) that takes advantage of coarse-to-fine information to improve deepfake detection. The proposed method involves extracting features from the frame, face, and key facial regions (i.e., lips, eyes, and nose) separately to explore the coarse-to-fine relationships. This approach enables us to capture inter-dependencies among facial regions using a channel-attention mechanism and deep triplet learning. We evaluated the proposed method on benchmark deepfake datasets in both intra-, inter-dataset, and inter-manipulation settings. The proposed method achieves an average AUC of 98.42\% on FF++, 79.80\% on CDF1, 85.34\% on CDF2, 89.41\% on DFD, 84.07\% on DFDC, 95.62\% on DTIM, 80.76\% on PDD, and 100\% on WLDR, respectively. The results demonstrate that our approach generalizes effectively and achieves promising performance to outperform the existing methods.
Abstract (translated)
多媒体数据（尤其是图像和视频）在监控、视觉交互、生物识别、证据收集和广告等多种应用中不可或缺。然而，业余或专业伪造者可能模拟这些数据制作深度伪造内容，往往出于诽谤目的。为应对这一挑战，已开发出多种取证方法以确保内容真实性。这些方法的有效性取决于其检测重点，而伪造手段的多样性也带来了相应挑战。本文分析了现有取证方法，发现每种方法通过聚焦特定面部区域（如整体画面、面部、嘴唇、眼睛或鼻子）在检测深度伪造痕迹方面具有独特优势。基于这些见解，我们提出了一种名为Face2Parts的新型混合方法，该方法采用分层特征表示（HFR），利用由粗到细的信息提升深度伪造检测效果。该方法分别从画面整体、面部及关键面部区域（即嘴唇、眼睛和鼻子）提取特征，以探索由粗到细的特征关联。此方法通过通道注意力机制和深度三元组学习，捕捉面部区域间的相互依赖关系。我们在多个基准深度伪造数据集上评估了该方法，涵盖域内、跨域及跨伪造手法场景。实验结果显示，该方法在FF++数据集上平均AUC达98.42%，CDF1数据集上为79.80%，CDF2数据集上为85.34%，DFD数据集上为89.41%，DFDC数据集上为84.07%，DTIM数据集上为95.62%，PDD数据集上为80.76%，WLDR数据集上达到100%。结果表明，该方法具有良好泛化能力，性能优于现有方法。
URL
https://arxiv.org/abs/2603.26036
