Abstract
Facial expression recognition is a crucial component in enhancing human-computer interaction and developing emotion-aware systems. Real-time detection and interpretation of facial expressions have become increasingly important for various applications, from user experience personalization to intelligent surveillance systems. This study presents a novel approach to real-time sequential facial expression recognition using deep learning and geometric features. The proposed method utilizes MediaPipe FaceMesh for rapid and accurate facial landmark detection. Geometric features, including Euclidean distances and angles, are extracted from these landmarks. Temporal dynamics are incorporated by analyzing feature differences between consecutive frames, enabling the detection of onset, apex, and offset phases of expressions. For classification, a ConvLSTM1D network followed by multilayer perceptron blocks is employed. The method's performance was evaluated on multiple publicly available datasets, including CK+, Oulu-CASIA (VIS and NIR), and MMI. Accuracies of 93%, 79%, 77%, and 68% were achieved respectively. Experiments with composite datasets were also conducted to assess the model's generalization capabilities. The approach demonstrated real-time applicability, processing approximately 165 frames per second on consumer-grade hardware. This research contributes to the field of facial expression analysis by providing a fast, accurate, and adaptable solution. The findings highlight the potential for further advancements in emotion-aware technologies and personalized user experiences, paving the way for more sophisticated human-computer interaction systems. To facilitate further research in this field, the complete source code for this study has been made publicly available on GitHub: this https URL.
Abstract (translated)
面部表情识别是增强人机交互和开发情感感知系统的关键组成部分。实时检测和解读面部表情对于各种应用变得越来越重要，从用户体验个性化到智能监控系统。本研究提出了一种使用深度学习和几何特征进行实时顺序面部表情识别的新方法。所提方法利用MediaPipe FaceMesh进行快速准确的面部标志点检测。通过这些标志点提取欧几里得距离和角度等几何特性。时间动态性通过分析连续帧之间特性的差异来融入，从而实现表情起始、顶峰和结束阶段的检测。分类过程中使用了ConvLSTM1D网络后接多层感知器模块。该方法在多个公开数据集上进行了性能评估，包括CK+、Oulu-CASIA（VIS和NIR）以及MMI，分别达到了93%、79%、77%和68%的准确率。为了评估模型的泛化能力，还对合成的数据集进行了实验。该方法展示了其实时适用性，在消费级硬件上每秒可处理约165帧。这项研究通过提供快速、准确且灵活的解决方案为面部表情分析领域做出了贡献。研究成果强调了在情感感知技术及个性化用户体验方面的进一步发展潜力，并为更高级的人机交互系统铺平道路。为了促进该领域的进一步研究，本研究所用的完整源代码已公开发布于GitHub：this https URL.
URL
https://arxiv.org/abs/2512.05669
