Abstract
Point-supervised Temporal Action Localization (PTAL) adopts a lightly frame-annotated paradigm (\textit{i.e.}, labeling only a single frame per action instance) to train a model to effectively locate action instances within untrimmed videos. Most existing approaches design the task head of models with only a point-supervised snippet-level classification, without explicit modeling of understanding temporal relationships among frames of an action. However, understanding the temporal relationships of frames is crucial because it can help a model understand how an action is defined and therefore benefits localizing the full frames of an action. To this end, in this paper, we design a multi-task learning framework that fully utilizes point supervision to boost the model's temporal understanding capability for action localization. Specifically, we design three self-supervised temporal understanding tasks: (i) Action Completion, (ii) Action Order Understanding, and (iii) Action Regularity Understanding. These tasks help a model understand the temporal consistency of actions across videos. To the best of our knowledge, this is the first attempt to explicitly explore temporal consistency for point supervision action localization. Extensive experimental results on four benchmark datasets demonstrate the effectiveness of the proposed method compared to several state-of-the-art approaches.
Abstract (translated)
点监督时间动作定位（PTAL）采用轻量级的帧标注范式（即，仅对每个动作实例标记一个帧），来训练模型以有效地在未剪辑的视频中定位动作实例。大多数现有方法设计了只有基于点监督的片段级别分类的任务头，并没有明确地建模理解动作帧之间的时序关系。然而，理解帧之间的时序关系至关重要，因为它可以帮助模型理解动作是如何定义的，并且有助于定位整个动作帧。为此，在本文中，我们设计了一个多任务学习框架，该框架充分利用了点监督以增强模型的时间理解能力来进行动作定位。具体来说，我们设计了三个自我监督的时间理解任务：（i）动作完成，（ii）动作顺序理解，和（iii）动作规律性理解。这些任务帮助模型理解动作在不同视频中的时间一致性。据我们所知，这是首次明确探索点监督动作定位中的时间一致性的尝试。我们在四个基准数据集上进行了广泛的实验，结果表明与几种最先进的方法相比，本文提出的方法是有效的。
URL
https://arxiv.org/abs/2602.05718
