Abstract
In videos containing spoofed faces, we may uncover the spoofing evidence based on either photometric or dynamic abnormality, even a combination of both. Prevailing face anti-spoofing (FAS) approaches generally concentrate on the single-frame scenario, however, purely photometric-driven methods overlook the dynamic spoofing clues that may be exposed over time. This may lead FAS systems to conclude incorrect judgments, especially in cases where it is easily distinguishable in terms of dynamics but challenging to discern in terms of photometrics. To this end, we propose the Graph Guided Video Vision Transformer (G$^2$V$^2$former), which combines faces with facial landmarks for photometric and dynamic feature fusion. We factorize the attention into space and time, and fuse them via a spatiotemporal block. Specifically, we design a novel temporal attention called Kronecker temporal attention, which has a wider receptive field, and is beneficial for capturing dynamic information. Moreover, we leverage the low-semantic motion of facial landmarks to guide the high-semantic change of facial expressions based on the motivation that regions containing landmarks may reveal more dynamic clues. Extensive experiments on nine benchmark datasets demonstrate that our method achieves superior performance under various scenarios. The codes will be released soon.
Abstract (translated)
在包含伪造脸的视频中，我们可能根据光电特征或动态异常来揭示伪造证据，甚至可以是两者之间的组合。预先存在的面部抗伪造（FAS）方法通常集中于单帧场景，然而，仅基于光学的伪造证据可能掩盖了随着时间的推移可能暴露的动态伪造线索。这可能导致FAS系统得出错误的判断，尤其是在 dynamics 上容易区分，但在 photometrics 上很难分辨的情况下。因此，我们提出了Graph Guided Video Vision Transformer (G$^2$V$^2$Transformer)，它结合了面部特征和面部标志以进行光电和动态特征融合。我们将注意力因素化到空间和时间，并通过一个spatiotemporal块将它们融合。具体来说，我们设计了一个名为Kronecker temporal attention的新型时间注意力，它具有更宽的接收范围，对于捕捉动态信息非常有利。此外，我们利用面部标志的低语义运动来指导根据动机，含有标志的区域可能揭示更动态的提示的脸上表情的语义变化。在九个基准数据集上的大量实验证明，我们的方法在各种情景下取得了卓越的性能。代码即将发布。
URL
https://arxiv.org/abs/2408.07675
