Abstract
Although facial landmark detection (FLD) has gained significant progress, existing FLD methods still suffer from performance drops on partially non-visible faces, such as faces with occlusions or under extreme lighting conditions or poses. To address this issue, we introduce ORFormer, a novel transformer-based method that can detect non-visible regions and recover their missing features from visible parts. Specifically, ORFormer associates each image patch token with one additional learnable token called the messenger token. The messenger token aggregates features from all but its patch. This way, the consensus between a patch and other patches can be assessed by referring to the similarity between its regular and messenger embeddings, enabling non-visible region identification. Our method then recovers occluded patches with features aggregated by the messenger tokens. Leveraging the recovered features, ORFormer compiles high-quality heatmaps for the downstream FLD task. Extensive experiments show that our method generates heatmaps resilient to partial occlusions. By integrating the resultant heatmaps into existing FLD methods, our method performs favorably against the state of the arts on challenging datasets such as WFLW and COFW.
Abstract (translated)
尽管面部特征点检测（FLD）已经取得了显著进展，现有的FLD方法在处理部分不可见的面部时仍然面临性能下降的问题，比如被遮挡或处于极端光照条件和姿态下的面部。为了解决这一问题，我们引入了ORFormer，这是一种基于Transformer的新方法，能够检测非可见区域并从可见部分恢复其缺失特征。具体来说，ORFormer将每个图像块标记与一个额外的可学习标记——信使标记相关联。该信使标记聚合除了它所属块外的所有其他特征。通过这种方式，可以通过参考常规嵌入和信使嵌入之间的相似性来评估一块与其他块之间的一致性，从而实现非可见区域识别。然后，我们的方法使用由信使标记聚集的特征恢复被遮挡的区块。借助恢复出的特征，ORFormer为下游FLD任务编译高质量的热图。广泛的实验表明，我们提出的方法生成的热图能够抵御部分遮挡的影响。通过将这些生成的热图整合到现有的FLD方法中，在WFLW和COFW等具有挑战性的数据集上，我们的方法表现优于现有技术。
URL
https://arxiv.org/abs/2412.13174
