Abstract
The identification of hazardous driving behaviors from in-cabin video streams is essential for enhancing road safety and supporting the detection of traffic violations and unsafe driver actions. However, current temporal action localization techniques often struggle to balance accuracy with computational efficiency. In this work, we develop and evaluate a temporal action localization framework tailored for driver monitoring scenarios, particularly suitable for periodic inspection settings such as transportation safety checkpoints or fleet management assessment systems. Our approach follows a two-stage pipeline that combines VideoMAE-based feature extraction with an Augmented Self-Mask Attention (AMA) detector, enhanced by a Spatial Pyramid Pooling-Fast (SPPF) module to capture multi-scale temporal features. Experimental results reveal a distinct trade-off between model capacity and efficiency. At the feature extraction stage, the ViT-Giant backbone delivers higher representations with 88.09% Top-1 test accuracy, while the ViT-based variant proves to be a practical alternative, achieving 82.55% accuracy with significantly lower computational fine-tuning costs (101.85 GFLOPs/segment compared to 1584.06 GFLOPs/segment for Giant). In the downstream localization task, the integration of SPPF consistently improves performance across all configurations. Notably, the ViT-Giant + SPPF model achieves a peak mAP of 92.67%, while the lightweight ViT-based configuration maintains robust results.
Abstract (translated)
基于车内视频流识别危险驾驶行为，对于提升道路安全以及辅助检测交通违规与不安全驾驶操作至关重要。然而，当前时序动作定位技术常难以兼顾准确性与计算效率。本研究开发并评估了一种专为驾驶员监控场景设计的时序动作定位框架，尤其适用于交通运输安全检查站或车队管理评估系统等周期性检查场景。该方法采用两阶段流程，结合基于VideoMAE的特征提取与增强自掩码注意力（AMA）检测器，并引入空间金字塔池化快速（SPPF）模块以捕获多尺度时序特征。实验结果表明，模型能力与效率之间存在明显权衡：在特征提取阶段，ViT-Giant主干网络以88.09%的Top-1测试准确率提供了更优表征，而基于ViT的变体则展现出实用性，以82.55%的准确率实现了显著更低的微调计算成本（每片段101.85 GFLOPs，而Giant为1584.06 GFLOPs）。在下游定位任务中，SPPF的集成持续提升了所有配置的性能。值得注意的是，ViT-Giant + SPPF模型达到了92.67%的峰值mAP，而轻量级基于ViT的配置也保持了稳健的结果。
URL
https://arxiv.org/abs/2603.21048
