Abstract
Weakly-supervised Temporal Action Localization (WTAL) has achieved notable success but still suffers from a lack of temporal annotations, leading to a performance and framework gap compared with fully-supervised methods. While recent approaches employ pseudo labels for training, three key challenges: generating high-quality pseudo labels, making full use of different priors, and optimizing training methods with noisy labels remain unresolved. Due to these perspectives, we propose PseudoFormer, a novel two-branch framework that bridges the gap between weakly and fully-supervised Temporal Action Localization (TAL). We first introduce RickerFusion, which maps all predicted action proposals to a global shared space to generate pseudo labels with better quality. Subsequently, we leverage both snippet-level and proposal-level labels with different priors from the weak branch to train the regression-based model in the full branch. Finally, the uncertainty mask and iterative refinement mechanism are applied for training with noisy pseudo labels. PseudoFormer achieves state-of-the-art WTAL results on the two commonly used benchmarks, THUMOS14 and ActivityNet1.3. Besides, extensive ablation studies demonstrate the contribution of each component of our method.
Abstract (translated)
弱监督时间动作定位（WTAL）已经取得了显著的成功，但仍因缺乏时间标注而与全监督方法在性能和框架方面存在差距。尽管最近的方法采用伪标签进行训练，但在生成高质量的伪标签、充分利用不同先验以及用噪声标签优化训练方法这三个关键挑战上仍存在问题。鉴于这些视角，我们提出了PseudoFormer，这是一种新颖的双分支框架，旨在弥合弱监督与全监督时间动作定位（TAL）之间的差距。 首先，我们引入RickerFusion，将所有预测的动作提案映射到一个全局共享空间中，以生成质量更好的伪标签。随后，利用弱分支中的片段级和提案级不同先验的标注来训练强分支中的回归模型。最后，应用不确定性掩码与迭代细化机制来进行噪声伪标签下的训练。 PseudoFormer在两个常用基准测试集THUMOS14和ActivityNet1.3上取得了当前最佳的WTAL结果。此外，广泛的消融研究展示了我们方法中每个组件的贡献。
URL
https://arxiv.org/abs/2504.14860
