Abstract
Pseudo-label learning methods have been widely applied in weakly-supervised temporal action localization. Existing works directly utilize weakly-supervised base model to generate instance-level pseudo-labels for training the fully-supervised detection head. We argue that the noise in pseudo-labels would interfere with the learning of fully-supervised detection head, leading to significant performance leakage. Issues with noisy labels include:(1) inaccurate boundary localization; (2) undetected short action clips; (3) multiple adjacent segments incorrectly detected as one segment. To target these issues, we introduce a two-stage noisy label learning strategy to harness every potential useful signal in noisy labels. First, we propose a frame-level pseudo-label generation model with a context-aware denoising algorithm to refine the boundaries. Second, we introduce an online-revised teacher-student framework with a missing instance compensation module and an ambiguous instance correction module to solve the short-action-missing and many-to-one problems. Besides, we apply a high-quality pseudo-label mining loss in our online-revised teacher-student framework to add different weights to the noisy labels to train more effectively. Our model outperforms the previous state-of-the-art method in detection accuracy and inference speed greatly upon the THUMOS14 and ActivityNet v1.2 benchmarks.
Abstract (translated)
伪标签学习方法在弱监督下的时序动作定位中得到了广泛应用。现有研究直接利用弱监督基础模型生成实例级伪标签来训练全监督检测头。我们认为，伪标签中的噪声会干扰全监督检测头的学习过程，导致显著的性能损失。带有噪声标签的问题包括：（1）边界定位不准确；（2）未检出短动作片段；（3）多个相邻段落被错误地识别为一个段落。 为了应对这些问题，我们提出了一种两阶段的噪声标签学习策略，以利用噪声标签中的每一个潜在有用信号。首先，我们提出了一个带有上下文感知降噪算法的帧级伪标签生成模型来优化边界定位。其次，我们引入了一个在线修正的教师-学生框架，并配备了一个缺失实例补偿模块和一个模棱两可实例校正模块，用于解决短动作片段丢失以及多个段落被错误地识别为单个段落的问题。此外，我们在改进后的教师-学生框架中应用了高质量伪标签挖掘损失函数，以对噪声标签赋予不同的权重，从而更有效地进行训练。 我们的模型在THUMOS14和ActivityNet v1.2基准测试上，在检测准确性和推理速度方面都显著优于之前的最先进方法。
URL
https://arxiv.org/abs/2501.11124
