Abstract
Online Temporal Action Localization (On-TAL) is a critical task that aims to instantaneously identify action instances in untrimmed streaming videos as soon as an action concludes -- a major leap from frame-based Online Action Detection (OAD). Yet, the challenge of detecting overlapping actions is often overlooked even though it is a common scenario in streaming videos. Current methods that can address concurrent actions depend heavily on class information, limiting their flexibility. This paper introduces ActionSwitch, the first class-agnostic On-TAL framework capable of detecting overlapping actions. By obviating the reliance on class information, ActionSwitch provides wider applicability to various situations, including overlapping actions of the same class or scenarios where class information is unavailable. This approach is complemented by the proposed "conservativeness loss", which directly embeds a conservative decision-making principle into the loss function for On-TAL. Our ActionSwitch achieves state-of-the-art performance in complex datasets, including Epic-Kitchens 100 targeting the challenging egocentric view and FineAction consisting of fine-grained actions.
Abstract (translated)
在线时元动作定位（On-TAL）是一个关键的任务，旨在在视频剪辑结束时立即识别完成动作实例，从基于帧的在线动作检测（OAD）中迈出了重要的一步。然而，即使这是流视频的常见情景，仍常常忽视了检测重叠动作的挑战。现有的方法在很大程度上依赖于类别信息，限制了其灵活性。本文介绍了ActionSwitch，第一个类无关的On-TAL框架，能够检测重叠动作。通过消除对类信息的依赖，ActionSwitch为各种情况提供了更广泛的适用性，包括相同类别的重叠动作或类信息不可用的情况。这种方法补充了提出的"容错损失"，将保守决策原则直接嵌入到On-TAL的损失函数中。我们的ActionSwitch在复杂的数据集上实现了最先进的性能，包括Epic-Kitchens 100（针对具有挑战性的自中心视角）和FineAction（由细粒度动作组成的场景）。
URL
https://arxiv.org/abs/2407.12987
