Abstract
Multi-object tracking (MOT) methods often rely on Intersection-over-Union (IoU) for association. However, this becomes unreliable when objects are similar or occluded. Also, computing IoU for segmentation masks is computationally expensive. In this work, we use segmentation masks to capture object shapes, but we do not compute segmentation IoU. Instead, we fuse depth and mask features and pass them through a compact encoder trained self-supervised. This encoder produces stable object representations, which we use as an additional similarity cue alongside bounding box IoU and re-identification features for matching. We obtain depth maps from a zero-shot depth estimator and object masks from a promptable visual segmentation model to obtain fine-grained spatial cues. Our MOT method is the first to use the self-supervised encoder to refine segmentation masks without computing masks IoU. MOT can be divided into joint detection-ReID (JDR) and tracking-by-detection (TBD) models. The latter are computationally more efficient. Experiments of our TBD method on challenging benchmarks with non-linear motion, occlusion, and crowded scenes, such as SportsMOT and DanceTrack, show that our method outperforms the TBD state-of-the-art on most metrics, while achieving competitive performance on simpler benchmarks with linear motion, such as MOT17.
Abstract (translated)
多目标跟踪（MOT）方法通常依赖于交并比（IoU）进行关联。然而，当对象相似或被遮挡时，这种方法变得不可靠。此外，计算分割掩码的IoU在计算上是昂贵的。在这项工作中，我们使用分割掩码来捕捉物体形状，但不计算分割IoU。相反，我们将深度和掩码特征融合，并通过一种经过自监督训练的小型编码器进行处理。这种编码器产生稳定的目标表示，我们在匹配时将其作为边界框IoU和重新识别特征之外的额外相似性线索使用。我们从零样本深度估计器获得深度图，并从可提示视觉分割模型中获取物体掩码，以获得细粒度的空间线索。 我们的MOT方法是第一个利用自监督编码器来改进分割掩码而不计算掩码IoU的方法。MOT可以分为联合检测-重识别（JDR）和基于检测的跟踪（TBD）模型两大类。后者在计算效率上更为高效。我们在具有非线性运动、遮挡以及拥挤场景等挑战性的基准测试，如SportsMOT和DanceTrack上的实验表明，我们的方法在大多数指标上优于现有的TBD最佳方法，并且在包含线性运动的简单基准测试（例如MOT17）中也取得了竞争性表现。
URL
https://arxiv.org/abs/2510.09878
