Abstract
Social interactions incorporate nonverbal signals to convey emotions alongside speech, including facial expressions and body gestures. Generative models have demonstrated promising results in creating full-body nonverbal animations synchronized with speech; however, evaluations using statistical metrics in 2D settings fail to fully capture user-perceived emotions, limiting our understanding of model effectiveness. To address this, we evaluate emotional 3D animation generative models within a Virtual Reality (VR) environment, emphasizing user-centric metrics emotional arousal realism, naturalness, enjoyment, diversity, and interaction quality in a real-time human-agent interaction scenario. Through a user study (N=48), we examine perceived emotional quality for three state of the art speech-driven 3D animation methods across two emotions happiness (high arousal) and neutral (mid arousal). Additionally, we compare these generative models against real human expressions obtained via a reconstruction-based method to assess both their strengths and limitations and how closely they replicate real human facial and body expressions. Our results demonstrate that methods explicitly modeling emotions lead to higher recognition accuracy compared to those focusing solely on speech-driven synchrony. Users rated the realism and naturalness of happy animations significantly higher than those of neutral animations, highlighting the limitations of current generative models in handling subtle emotional states. Generative models underperformed compared to reconstruction-based methods in facial expression quality, and all methods received relatively low ratings for animation enjoyment and interaction quality, emphasizing the importance of incorporating user-centric evaluations into generative model development. Finally, participants positively recognized animation diversity across all generative models.
Abstract (translated)
社交互动通过言语与非语言信号（如面部表情和肢体手势）共同传递情感。生成模型在创建与语音同步的全身非语言动画方面已展现出前景；然而，基于二维场景的统计指标评估未能充分捕捉用户感知的情感，制约了我们对模型有效性的理解。为解决这一问题，我们在虚拟现实（VR）环境中评估情感三维动画生成模型，重点关注实时人机交互场景中以用户为中心的指标：情感唤醒真实感、自然度、愉悦感、多样性及交互质量。通过一项用户研究（N=48），我们考察了三种最先进的语音驱动三维动画方法在两种情感（高兴[高唤醒]与中性[中唤醒]）下的感知情感质量。此外，我们将这些生成模型与通过基于重建方法获取的真实人类表情进行对比，以评估其优势与局限，以及它们在多大程度上复现了真实人类的面部与身体表达。研究结果表明，显式建模情感的方法相较于仅关注语音同步的方法具有更高的识别准确率。用户对高兴动画的真实感与自然度评分显著高于中性动画，凸显了当前生成模型在处理微妙情感状态时的局限性。在面部表情质量上，生成模型表现低于基于重建的方法；所有方法在动画愉悦感与交互质量上均获得相对低分，强调了将用户为中心评估纳入生成模型开发的重要性。最后，参与者对所有生成模型的动画多样性均给予积极认可。
URL
https://arxiv.org/abs/2512.16081
