Abstract
Recent advances in deep learning have significantly improved facial landmark detection. However, existing facial landmark detection datasets often define different numbers of landmarks, and most mainstream methods can only be trained on a single dataset. This limits the model generalization to different datasets and hinders the development of a unified model. To address this issue, we propose Proto-Former, a unified, adaptive, end-to-end facial landmark detection framework that explicitly enhances dataset-specific facial structural representations (i.e., prototype). Proto-Former overcomes the limitations of single-dataset training by enabling joint training across multiple datasets within a unified architecture. Specifically, Proto-Former comprises two key components: an Adaptive Prototype-Aware Encoder (APAE) that performs adaptive feature extraction and learns prototype representations, and a Progressive Prototype-Aware Decoder (PPAD) that refines these prototypes to generate prompts that guide the model's attention to key facial regions. Furthermore, we introduce a novel Prototype-Aware (PA) loss, which achieves optimal path finding by constraining the selection weights of prototype experts. This loss function effectively resolves the problem of prototype expert addressing instability during multi-dataset training, alleviates gradient conflicts, and enables the extraction of more accurate facial structure features. Extensive experiments on widely used benchmark datasets demonstrate that our Proto-Former achieves superior performance compared to existing state-of-the-art methods. The code is publicly available at: this https URL.
Abstract (translated)
近期在深度学习领域取得的进步显著提升了面部特征点检测的精度。然而，现有的面部特征点检测数据集往往定义了不同数量的特征点，并且大多数主流方法只能在一个单一的数据集上进行训练。这限制了模型对不同数据集的泛化能力，并阻碍了一体化模型的发展。为解决这一问题，我们提出了Proto-Former，这是一种统一、自适应的一站式面部特征点检测框架，它明确增强了特定于各个数据集的面部结构表示（即原型）。通过在统一架构中实现跨多个数据集的同时训练，Proto-Former克服了单一数据集训练的局限性。 具体而言，Proto-Former包括两个关键组成部分：自适应原型感知编码器（APAE），该编码器执行自适应特征提取，并学习原型表示；渐进式原型感知解码器（PPAD），它对这些原型进行精炼以生成引导模型注意特定面部区域的提示。此外，我们引入了一种新颖的原型感知（PA）损失函数，通过约束原型专家的选择权重来实现最佳路径搜索。这种损失函数有效地解决了多数据集训练期间原型专家地址不稳定的难题，减轻了梯度冲突，并使更准确地提取面部结构特征成为可能。 在广泛使用的基准数据集中进行的大量实验表明，我们的Proto-Former相较于现有最先进的方法具有优越的表现。代码可在以下网址公开获取：this https URL.
URL
https://arxiv.org/abs/2510.15338
