Abstract
Person re-identification (re-ID) via 3D skeleton data is a challenging task with significant value in many scenarios. Existing skeleton-based methods typically assume virtual motion relations between all joints, and adopt average joint or sequence representations for learning. However, they rarely explore key body structure and motion such as gait to focus on more important body joints or limbs, while lacking the ability to fully mine valuable spatial-temporal sub-patterns of skeletons to enhance model learning. This paper presents a generic Motif guided graph transformer with Combinatorial skeleton prototype learning (MoCos) that exploits structure-specific and gait-related body relations as well as combinatorial features of skeleton graphs to learn effective skeleton representations for person re-ID. In particular, motivated by the locality within joints' structure and the body-component collaboration in gait, we first propose the motif guided graph transformer (MGT) that incorporates hierarchical structural motifs and gait collaborative motifs, which simultaneously focuses on multi-order local joint correlations and key cooperative body parts to enhance skeleton relation learning. Then, we devise the combinatorial skeleton prototype learning (CSP) that leverages random spatial-temporal combinations of joint nodes and skeleton graphs to generate diverse sub-skeleton and sub-tracklet representations, which are contrasted with the most representative features (prototypes) of each identity to learn class-related semantics and discriminative skeleton representations. Extensive experiments validate the superior performance of MoCos over existing state-of-the-art models. We further show its generality under RGB-estimated skeletons, different graph modeling, and unsupervised scenarios.
Abstract (translated)
基于3D骨骼数据的人体再识别（re-ID）是一项在许多场景中具有重要价值的挑战性任务。现有的基于骨架的方法通常假设所有关节之间的虚拟运动关系，并采用平均关节或序列表示进行学习。然而，它们很少探索关键的身体结构和运动，例如步态，以更专注于重要的身体关节或四肢，同时缺乏充分挖掘骨骼有价值的空间-时间子模式的能力来增强模型的学习效果。本文提出了一种通用的动机引导图变换器与组合骨架原型学习（MoCos）方法，该方法利用特定于结构和与步态相关的身体关系以及骨架图的组合特征，以学习有效的骨架表示用于人体再识别。特别地，在关节结构内的局部性和步态中的身体部件协作的启发下，我们首先提出了动机引导图变换器（MGT），它结合了分层结构模式和步态协作模式，同时关注多阶局部关节关联和关键合作的身体部位以增强骨骼关系学习。然后，我们设计了一种组合骨架原型学习（CSP）方法，利用关节节点和骨架图的随机空间-时间组合来生成多样化的子骨架和子轨迹表示，并将这些表示与每个身份最具代表性的特征（原型）进行对比，从而学习类别相关的语义和辨别性骨骼表示。广泛的实验验证了MoCos在现有最先进的模型中的优越性能。我们进一步展示了它在基于RGB估算的骨架、不同的图建模以及无监督场景下的通用性。
URL
https://arxiv.org/abs/2412.09044
