Abstract
Multi-face deepfake videos are becoming increasingly prevalent, often appearing in natural social settings that challenge existing detection methods. Most current approaches excel at single-face detection but struggle in multi-face scenarios, due to a lack of awareness of crucial contextual cues. In this work, we develop a novel approach that leverages human cognition to analyze and defend against multi-face deepfake videos. Through a series of human studies, we systematically examine how people detect deepfake faces in social settings. Our quantitative analysis reveals four key cues humans rely on: scene-motion coherence, inter-face appearance compatibility, interpersonal gaze alignment, and face-body consistency. Guided by these insights, we introduce \textsf{HICOM}, a novel framework designed to detect every fake face in multi-face scenarios. Extensive experiments on benchmark datasets show that \textsf{HICOM} improves average accuracy by 3.3\% in in-dataset detection and 2.8\% under real-world perturbations. Moreover, it outperforms existing methods by 5.8\% on unseen datasets, demonstrating the generalization of human-inspired cues. \textsf{HICOM} further enhances interpretability by incorporating an LLM to provide human-readable explanations, making detection results more transparent and convincing. Our work sheds light on involving human factors to enhance defense against deepfakes.
Abstract (translated)
多面深度伪造视频正变得越来越普遍，通常出现在自然社交环境中，这使得现有的检测方法难以应对。大多数当前的方法在单人脸检测方面表现出色，但在处理多个人脸的场景时却遇到困难，因为它们缺乏对关键上下文线索的理解。在这项工作中，我们开发了一种新的方法，该方法利用人类认知来分析和防御多面深度伪造视频。通过一系列的人类研究，我们系统地检查了人们在社交环境中如何检测深度伪造人脸的关键线索。定量分析揭示了四个关键因素：场景运动一致性、人脸间外观兼容性、人际注视对齐以及面部与身体的一致性。 基于这些见解，我们引入了一个新的框架\textsf{HICOM}，旨在识别多面场景中每一个假脸。在基准数据集上的广泛实验表明，\textsf{HICOM}使内部分类检测的平均准确率提高了3.3%，在现实世界扰动下的准确性提高了2.8%。此外，在未见过的数据集中，它比现有方法高出5.8%，展示了人类启发式线索的一般化能力。通过整合LLM（大型语言模型）来提供可读性解释，\textsf{HICOM}进一步增强了检测结果的透明度和说服力。 我们的工作为利用人类因素增强对深度伪造防御的研究提供了新的思路。
URL
https://arxiv.org/abs/2507.14807
