Abstract
Temporal Action Localization (TAL) remains a fundamental challenge in video understanding, aiming to identify the start time, end time, and category of all action instances within untrimmed videos. While recent single-stage, anchor-free models like ActionFormer have set a high standard by leveraging Transformers for temporal reasoning, they often struggle with two persistent issues: the precise localization of actions with ambiguous or "fuzzy" temporal boundaries and the effective fusion of multi-scale contextual information. In this paper, we introduce the Temporal Boundary Transformer (TBT-Former), a new architecture that directly addresses these limitations. TBT-Former enhances the strong ActionFormer baseline with three core contributions: (1) a higher-capacity scaled Transformer backbone with an increased number of attention heads and an expanded Multi-Layer Perceptron (MLP) dimension for more powerful temporal feature extraction; (2) a cross-scale feature pyramid network (FPN) that integrates a top-down pathway with lateral connections, enabling richer fusion of high-level semantics and low-level temporal details; and (3) a novel boundary distribution regression head. Inspired by the principles of Generalized Focal Loss (GFL), this new head recasts the challenging task of boundary regression as a more flexible probability distribution learning problem, allowing the model to explicitly represent and reason about boundary uncertainty. Within the paradigm of Transformer-based architectures, TBT-Former advances the formidable benchmark set by its predecessors, establishing a new level of performance on the highly competitive THUMOS14 and EPIC-Kitchens 100 datasets, while remaining competitive on the large-scale ActivityNet-1.3. Our code is available at this https URL
Abstract (translated)
时序动作定位（TAL）在视频理解中仍是一个基本挑战，目标是识别未剪辑视频内所有动作实例的开始时间、结束时间和类别。虽然近期的一些单阶段无锚点模型，例如ActionFormer，通过利用Transformer进行时间推理已设立了高标准，但它们经常面临两个持久性问题：难以精确定位具有模糊或“模棱两可”时间边界的动作以及有效融合多尺度上下文信息的挑战。本文介绍了Temporal Boundary Transformer (TBT-Former)，这是一种新的架构，直接解决了这些限制。通过三个核心贡献增强了强大的ActionFormer基线模型： 1. 更强大容量的时间Transformer骨干网络，具有增加的关注头数量和扩展的多层感知机（MLP）维度，以实现更强大的时间特征提取； 2. 跨尺度特征金字塔网络 (FPN)，该网络集成了自上而下的路径与横向连接，使高级语义信息和低级时间细节的融合更加丰富； 3. 一种新颖的时间边界分布回归头。受到广义焦点损失（GFL）原理的启发，这个新的头部将具有挑战性的边界回归任务重新定义为一个更灵活的概率分布学习问题，允许模型明确表示并推理关于边界的不确定性。 在基于Transformer架构的框架内，TBT-Former在其前辈设定的强大基准上取得了进展，在竞争激烈的THUMOS14和EPIC-Kitchens 100数据集上建立了新的性能水平，并且在大规模的ActivityNet-1.3上保持了竞争力。我们的代码可在该链接获取：[提供的URL]
URL
https://arxiv.org/abs/2512.01298
