Abstract
Temporal Action Localization (TAL) aims to detect the start and end timestamps of actions in a video. However, the training of TAL models requires a substantial amount of manually annotated data. Data programming is an efficient method to create training labels with a series of human-defined labeling functions. However, its application in TAL faces difficulties of defining complex actions in the context of temporal video frames. In this paper, we propose ProTAL, a drag-and-link video programming framework for TAL. ProTAL enables users to define \textbf{key events} by dragging nodes representing body parts and objects and linking them to constrain the relations (direction, distance, etc.). These definitions are used to generate action labels for large-scale unlabelled videos. A semi-supervised method is then employed to train TAL models with such labels. We demonstrate the effectiveness of ProTAL through a usage scenario and a user study, providing insights into designing video programming framework.
Abstract (translated)
时序动作定位（Temporal Action Localization，TAL）的目标是在视频中检测动作的起始和结束时间戳。然而，训练TAL模型需要大量的手动标注数据。数据编程是一种通过一系列由人类定义的标签函数来创建训练标签的有效方法。然而，在TAL中的应用面临着如何在时序视频帧背景下定义复杂动作的挑战。为此，本文提出了ProTAL，这是一种用于TAL的拖拽链接视频编程框架。在ProTAL中，用户可以通过拖动代表身体部位和物体的节点，并将它们连接起来以约束关系（方向、距离等），来定义**关键事件**。这些定义被用来为大规模未标注视频生成动作标签。接着使用半监督方法利用这些标签训练TAL模型。我们通过一个应用场景和用户研究展示了ProTAL的有效性，这为我们设计视频编程框架提供了见解。 这段描述介绍了ProTAL框架的设计思路及其在时序动作定位任务中的应用价值。ProTAL通过允许用户定义关键事件来简化大规模未标注视频的动作标签生成过程，并且该方法利用了半监督学习技术进一步训练模型。
URL
https://arxiv.org/abs/2505.17555
