Abstract
Micro-actions, fleeting and low-amplitude motions, such as glances, nods, or minor posture shifts, carry rich social meaning but remain difficult for current action recognition models to recognize due to their subtlety, short duration, and high inter-class ambiguity. In this paper, we introduce B-MoE, a Body-part-aware Mixture-of-Experts framework designed to explicitly model the structured nature of human motion. In B-MoE, each expert specializes in a distinct body region (head, body, upper limbs, lower limbs), and is based on the lightweight Macro-Micro Motion Encoder (M3E) that captures long-range contextual structure and fine-grained local motion. A cross-attention routing mechanism learns inter-region relationships and dynamically selects the most informative regions for each micro-action. B-MoE uses a dual-stream encoder that fuses these region-specific semantic cues with global motion features to jointly capture spatially localized cues and temporally subtle variations that characterize micro-actions. Experiments on three challenging benchmarks (MA-52, SocialGesture, and MPII-GroupInteraction) show consistent state-of-theart gains, with improvements in ambiguous, underrepresented, and low amplitude classes.
Abstract (translated)
微动作，即那些转瞬即逝、幅度细微的动作（如瞥视、点头或轻微的姿态调整），蕴含着丰富的社交意义。然而，由于其细微性、持续时间短以及类间模糊性高，当前的动作识别模型难以有效识别这类动作。本文提出B-MoE（Body-part-aware Mixture-of-Experts）框架，该框架通过显式建模人体运动的结构化特性进行设计。在B-MoE中，每个专家专注于不同的身体区域（头部、躯干、上肢、下肢），并基于轻量级的宏观-微观运动编码器（M3E），该编码器能够捕捉长程上下文结构与细粒度局部运动。交叉注意力路由机制可学习区域间关系，并为每个微动作动态选择最具信息量的区域。B-MoE采用双流编码器，将这些区域特定的语义线索与全局运动特征相融合，共同捕捉微动作所特有的空间局部化线索与时间细微变化。在三个具有挑战性的基准数据集（MA-52、SocialGesture和MPII-GroupInteraction）上的实验表明，该方法持续取得最先进的性能提升，尤其在模糊、代表性不足及低幅度类别上表现显著。
URL
https://arxiv.org/abs/2603.24245
