Abstract
Automated Facial Expression Recognition (FER) is challenging due to intra-class variations and inter-class similarities. FER can be especially difficult when facial expressions reflect a mixture of various emotions (aka compound expressions). Existing FER datasets, such as AffectNet, provide discrete emotion labels (hard-labels), where a single category of emotion is assigned to an expression. To alleviate inter- and intra-class challenges, as well as provide a better facial expression descriptor, we propose a new approach to create FER datasets through a labeling method in which an image is labeled with more than one emotion (called soft-labels), each with different confidences. Specifically, we introduce the notion of soft-labels for facial expression datasets, a new approach to affective computing for more realistic recognition of facial expressions. To achieve this goal, we propose a novel methodology to accurately calculate soft-labels: a vector representing the extent to which multiple categories of emotion are simultaneously present within a single facial expression. Finding smoother decision boundaries, enabling multi-labeling, and mitigating bias and imbalanced data are some of the advantages of our proposed method. Building upon AffectNet, we introduce AffectNet+, the next-generation facial expression dataset. This dataset contains soft-labels, three categories of data complexity subsets, and additional metadata such as age, gender, ethnicity, head pose, facial landmarks, valence, and arousal. AffectNet+ will be made publicly accessible to researchers.
Abstract (translated)
自动化面部表情识别（FER）具有挑战性，主要由于类内变化和类间相似性的存在。当面部表情反映多种情感的混合时（即复合表达），FER尤其困难。现有的FER数据集，如AffectNet，提供离散的情感标签（硬标签），其中一种情绪类别被分配给一个表情。为了缓解类内和类间的挑战，并提供更好的面部表情描述符，我们提出了一种新的方法来创建FER数据集：通过标注方法对一张图片进行多于一种情感的标注（称为软标签），每种情感带有不同的置信度。具体来说，我们引入了面部表情数据集中软标签的概念，这是一种更为现实的情感计算新方法，用于面部表情识别。为了实现这一目标，我们提出了一种新的方法来准确计算软标签：一个向量表示在单个面部表情中同时存在多种情绪的程度。我们的提议方法具有发现平滑决策边界、支持多标注以及减轻偏差和数据不平衡的优势。基于AffectNet，我们引入了下一代面部表情数据集AffectNet+。该数据集包含软标签、三个类别的数据复杂度子集以及额外的元数据如年龄、性别、种族、头部姿态、面部特征点、效价和唤醒水平。AffectNet+将向研究人员公开提供访问权限。
URL
https://arxiv.org/abs/2410.22506
