Abstract
The present few-shot temporal action localization model can't handle the situation where videos contain multiple action instances. So the purpose of this paper is to achieve manifold action instances localization in a lengthy untrimmed query video using limited trimmed support videos. To address this challenging problem effectively, we proposed a novel solution involving a spatial-channel relation transformer with probability learning and cluster refinement. This method can accurately identify the start and end boundaries of actions in the query video, utilizing only a limited number of labeled videos. Our proposed method is adept at capturing both temporal and spatial contexts to effectively classify and precisely locate actions in videos, enabling a more comprehensive utilization of these crucial details. The selective cosine penalization algorithm is designed to suppress temporal boundaries that do not include action scene switches. The probability learning combined with the label generation algorithm alleviates the problem of action duration diversity and enhances the model's ability to handle fuzzy action boundaries. The interval cluster can help us get the final results with multiple instances situations in few-shot temporal action localization. Our model achieves competitive performance through meticulous experimentation utilizing the benchmark datasets ActivityNet1.3 and THUMOS14. Our code is readily available at this https URL.
Abstract (translated)
目前,这种少数shot时空动作局部定位模型无法处理视频包含多个动作实例的情况。因此,本文的目的是使用有限的剪辑支持视频在长篇未剪辑查询视频中实现多实例动作实例的局部定位,以解决这一具有挑战性的问题。为了有效地解决这个具有挑战性的问题,我们提出了一个新方法,涉及空间通道关系Transformer、概率学习和聚类 refinement。这种方法可以准确地确定查询视频中的动作开始和结束边界,只需使用很少量的标记视频。我们所提出的方法擅长捕捉时空上下文,从而有效地对视频进行分类和精确定位动作,有助于更全面地利用这些关键信息。选择性余弦惩罚算法旨在抑制不包含动作场景切换的时空边界。概率学习和标签生成算法相结合减轻了动作持续时间多样性的问题,提高了模型处理模糊动作边界的能力。区间聚类可以帮助我们在几 shot时空动作局部定位中处理多个实例的情况。我们的模型通过仔细实验利用基准数据集ActivityNet1.3和THUMOS14获得了竞争力的性能。我们的代码可通过此链接获取。
URL
https://arxiv.org/abs/2408.13765
