Abstract
This paper expands the cascaded network branch of the autoencoder-based multi-task learning (MTL) framework for dynamic facial expression recognition, namely Multi-Task Cascaded Autoencoder for Dynamic Facial Expression Recognition (MTCAE-DFER). MTCAE-DFER builds a plug-and-play cascaded decoder module, which is based on the Vision Transformer (ViT) architecture and employs the decoder concept of Transformer to reconstruct the multi-head attention module. The decoder output from the previous task serves as the query (Q), representing local dynamic features, while the Video Masked Autoencoder (VideoMAE) shared encoder output acts as both the key (K) and value (V), representing global dynamic features. This setup facilitates interaction between global and local dynamic features across related tasks. Additionally, this proposal aims to alleviate overfitting of complex large model. We utilize autoencoder-based multi-task cascaded learning approach to explore the impact of dynamic face detection and dynamic face landmark on dynamic facial expression recognition, which enhances the model's generalization ability. After we conduct extensive ablation experiments and comparison with state-of-the-art (SOTA) methods on various public datasets for dynamic facial expression recognition, the robustness of the MTCAE-DFER model and the effectiveness of global-local dynamic feature interaction among related tasks have been proven.
Abstract (translated)
本文扩展了基于自动编码器的多任务学习（MTL）框架中的级联网络分支，以实现动态面部表情识别。该方法称为“动态面部表情识别的多任务级联自动编码器”(MTCAE-DFER)。MTCAE-DFER构建了一个即插即用的级联解码模块，基于Vision Transformer (ViT)架构，并采用Transformer的解码概念来重构多头注意力机制。来自前一任务的解码器输出作为查询(Q)，表示局部动态特征；而Video Masked Autoencoder (VideoMAE)共享编码器的输出则充当键(K)和值(V)，代表全局动态特征。这种设置促进了相关任务中全局与局部动态特征之间的交互。 此外，该提案旨在缓解复杂大型模型的过拟合问题。我们采用基于自动编码器的多任务级联学习方法来探索动态面部检测和动态面部关键点对动态面部表情识别的影响，从而提高模型的泛化能力。经过广泛的消融实验以及与各种公开数据集上最新的动态面部表情识别方法进行对比后，MTCAE-DFER模型的鲁棒性以及相关任务中全局局部动态特征交互的有效性已经得到证实。
URL
https://arxiv.org/abs/2412.18988
