Abstract
This paper proposes a new effective and efficient plug-and-play backbone for video-based person re-identification (ReID). Conventional video-based ReID methods typically use CNN or transformer backbones to extract deep features for every position in every sampled video frame. Here, we argue that this exhaustive feature extraction could be unnecessary, since we find that different frames in a ReID video often exhibit small differences and contain many similar regions due to the relatively slight movements of human beings. Inspired by this, a more selective, efficient paradigm is explored in this paper. Specifically, we introduce a patch selection mechanism to reduce computational cost by choosing only the crucial and non-repetitive patches for feature extraction. Additionally, we present a novel network structure that generates and utilizes pseudo frame global context to address the issue of incomplete views resulting from sparse inputs. By incorporating these new designs, our backbone can achieve both high performance and low computational cost. Extensive experiments on multiple datasets show that our approach reduces the computational cost by 74\% compared to ViT-B and 28\% compared to ResNet50, while the accuracy is on par with ViT-B and outperforms ResNet50 significantly.
Abstract (translated)
本文提出了一种新的有效且高效的基于视频的人员重新识别（ReID）插件式骨干网络。传统的基于视频的ReID方法通常使用CNN或变压器骨干来提取每个采样视频帧中每个位置的深层特征。在这里，我们指出这种详尽无遗的特征提取可能是不必要的，因为我们发现，在一个ReID视频中的不同帧之间往往存在细微差异，并且由于人类相对微小的动作而包含许多相似区域。受到这一观察的启发，本文探讨了一种更为选择性和高效的范式。 具体来说，我们引入了一个补丁选择机制，通过只选择关键且非重复的补丁来进行特征提取以减少计算成本。此外，我们提出了一种新的网络结构，该结构生成并利用伪帧全局上下文来解决由于稀疏输入导致的视角不完整问题。 通过结合这些新设计，我们的骨干网能够实现高性能与低计算成本的良好平衡。在多个数据集上的广泛实验表明，相比于ViT-B和ResNet50，我们的方法可以将计算成本分别降低74% 和 28%，同时精度可媲美于ViT-B，并且显著优于ResNet50。
URL
https://arxiv.org/abs/2501.16811
