Abstract
This paper presents VisioPhysioENet, a novel multimodal system that leverages visual cues and physiological signals to detect learner engagement. It employs a two-level approach for visual feature extraction using the Dlib library for facial landmark extraction and the OpenCV library for further estimations. This is complemented by extracting physiological signals using the plane-orthogonal-to-skin method to assess cardiovascular activity. These features are integrated using advanced machine learning classifiers, enhancing the detection of various engagement levels. We rigorously evaluate VisioPhysioENet on the DAiSEE dataset, where it achieves an accuracy of 63.09%, demonstrating a superior ability to discern various levels of engagement compared to existing methodologies. The proposed system's code can be accessed at this https URL.
Abstract (translated)
本文介绍了一种名为VisioPhysioENet的多模态系统，该系统利用视觉提示和生理信号来检测学习者的参与度。它采用了一种双级方法，基于Dlib库进行面部关键点提取，并使用OpenCV库进行进一步估计。这通过使用平面与皮肤垂直的方法提取生理信号来评估心血管活动。这些特征使用先进的机器学习分类器进行整合，增强了各种参与度的检测。我们在DAiSEE数据集上对VisioPhysioENet进行了严格的评估，其精度为63.09%，表明其比现有方法更能够辨别各种参与度水平。该系统的代码可以在此链接访问：<https://www.academia.edu/39411041/VisioPhysioENet>
URL
https://arxiv.org/abs/2409.16126
