Abstract
Generic Event Boundary Detection (GEBD) aims to interpret long-form videos through the lens of human perception. However, current GEBD methods require processing complete video frames to make predictions, unlike humans processing data online and in real-time. To bridge this gap, we introduce a new task, Online Generic Event Boundary Detection (On-GEBD), aiming to detect boundaries of generic events immediately in streaming videos. This task faces unique challenges of identifying subtle, taxonomy-free event changes in real-time, without the access to future frames. To tackle these challenges, we propose a novel On-GEBD framework, Estimator, inspired by Event Segmentation Theory (EST) which explains how humans segment ongoing activity into events by leveraging the discrepancies between predicted and actual information. Our framework consists of two key components: the Consistent Event Anticipator (CEA), and the Online Boundary Discriminator (OBD). Specifically, the CEA generates a prediction of the future frame reflecting current event dynamics based solely on prior frames. Then, the OBD measures the prediction error and adaptively adjusts the threshold using statistical tests on past errors to capture diverse, subtle event transitions. Experimental results demonstrate that Estimator outperforms all baselines adapted from recent online video understanding models and achieves performance comparable to prior offline-GEBD methods on the Kinetics-GEBD and TAPOS datasets.
Abstract (translated)
通用事件边界检测（GEBD）旨在通过人类感知的角度来解释长视频内容。然而，当前的GEBD方法需要处理完整的视频帧才能做出预测，而人类则是实时在线地处理数据。为弥合这一差距，我们引入了一个新的任务——在线通用事件边界检测（On-GEBD），其目标是在流媒体视频中立即检测通用事件的边界。这个任务面临独特的挑战，即在没有未来帧访问权限的情况下，实时识别细微、无类别定义的事件变化。 为了应对这些挑战，我们提出了一种新颖的On-GEBD框架——Estimator，该框架受事件分割理论（EST）启发，解释人类如何通过利用预测信息与实际信息之间的差异来将正在进行的活动划分为不同的事件。我们的框架包含两个关键组成部分：一致事件预期器（CEA）和在线边界辨别器（OBD）。具体来说，CEA仅基于先前帧生成对未来帧的预测，反映当前事件的动力学；随后，OBD测量预测误差，并通过统计测试在过去的错误上自适应地调整阈值以捕捉多样且细微的事件过渡。 实验结果表明，Estimator优于所有从最近在线视频理解模型中改编而来的基线方法，在Kinetics-GEBD和TAPOS数据集上的性能与先前的离线GEBD方法相当。
URL
https://arxiv.org/abs/2510.06855
