Abstract
In this paper, we propose a new instance-level human-object interaction detection task on videos called ST-HOID, which aims to distinguish fine-grained human-object interactions (HOIs) and the trajectories of subjects and objects. It is motivated by the fact that HOI is crucial for human-centric video content understanding. To solve ST-HOID, we propose a novel method consisting of an object trajectory detection module and an interaction reasoning module. Furthermore, we construct the first dataset named VidOR-HOID for ST-HOID evaluation, which contains 10,831 spatial-temporal HOI instances. We conduct extensive experiments to evaluate the effectiveness of our method. The experimental results demonstrate that our method outperforms the baselines generated by the state-of-the-art methods of image human-object interaction detection, video visual relation detection and video human-object interaction recognition.
Abstract (translated)
在本文中，我们提出了一项新的视频级实例人类-物体交互检测任务，称为ST-HOID（空间时间人类-对象交互检测），该任务旨在区分细粒度的人类-物体互动(HOIs)以及主体和客体的轨迹。这项研究受到这样一个事实的启发：即HOI对于以人为中心的视频内容理解至关重要。为了解决ST-HOID问题，我们提出了一种新颖的方法，包括一个对象轨迹检测模块和一个交互推理模块。此外，为了评估ST-HOID，我们构建了首个名为VidOR-HOID的数据集，该数据集中包含10,831个空间时间HOI实例。我们进行了广泛的实验来评估我们的方法的有效性。实验结果表明，相较于由最先进的图像人类-物体互动检测、视频视觉关系检测和视频人类-物体互动识别等领域的基线方法生成的结果，我们的方法表现出更佳的性能。
URL
https://arxiv.org/abs/2508.17270
