Abstract
Although state-of-the-art classifiers for facial expression recognition (FER) can achieve a high level of accuracy, they lack interpretability, an important feature for end-users. Experts typically associate spatial action units (AUs) from a codebook to facial regions for the visual interpretation of expressions. In this paper, the same expert steps are followed. A new learning strategy is proposed to explicitly incorporate AU cues into classifier training, allowing to train deep interpretable models. During training, this AU codebook is used, along with the input image expression label, and facial landmarks, to construct a AU heatmap that indicates the most discriminative image regions of interest w.r.t the facial expression. This valuable spatial cue is leveraged to train a deep interpretable classifier for FER. This is achieved by constraining the spatial layer features of a classifier to be correlated with AU heatmaps. Using a composite loss, the classifier is trained to correctly classify an image while yielding interpretable visual layer-wise attention correlated with AU maps, simulating the expert decision process. Our strategy only relies on image class expression for supervision, without additional manual annotations. Our new strategy is generic, and can be applied to any deep CNN- or transformer-based classifier without requiring any architectural change or significant additional training time. Our extensive evaluation on two public benchmarks RAF-DB, and AffectNet datasets shows that our proposed strategy can improve layer-wise interpretability without degrading classification performance. In addition, we explore a common type of interpretable classifiers that rely on class activation mapping (CAM) methods, and show that our approach can also improve CAM interpretability.
Abstract (translated)
尽管最先进的面部表情识别（FER）分类器可以实现高水平的准确性，但它们缺乏可解释性，这对于最终用户来说非常重要。专家通常将代码库中的空间动作单元（AUs）与表情进行视觉解释。在本文中，遵循了相同的专家步骤。提出了一种新的学习策略，将AU提示明确地融入分类器训练中，以训练深度可解释的模型。在训练过程中，使用了新的AU代码本和表情图像标签以及面部关键点来构建AU热力图，表示与面部表情最相关的图像区域。这个有价值的空间线索被用于训练FER的深度可解释分类器。通过约束分类器的空间层特征与AU热力图相关，实现了训练分类器正确分类图像，并产生与AU地图相关的可解释视觉层级的关注，模拟了专家的决策过程。我们的策略仅依赖于图像类别的表达进行监督，没有额外的手动注释。我们的新策略是通用的，可以应用于任何基于深度卷积神经网络（CNN）或Transformer的分类器，而不需要进行任何架构上的改变或额外的训练时间。我们对两个公开基准数据集RAF-DB和AffectNet进行的大量评估显示，与我们的建议的策略相比，可以在不降低分类性能的情况下提高层间可解释性。此外，我们还研究了依赖于分类激活映射（CAM）方法的常见可解释分类器，并证明了我们的方法也可以提高CAM的可解释性。
URL
https://arxiv.org/abs/2410.01848
