Abstract
Human-centric perception is the core of diverse computer vision tasks and has been a long-standing research focus. However, previous research studied these human-centric tasks individually, whose performance is largely limited to the size of the public task-specific datasets. Recent human-centric methods leverage the additional modalities, e.g., depth, to learn fine-grained semantic information, which limits the benefit of pretraining models due to their sensitivity to camera views and the scarcity of RGB-D data on the Internet. This paper improves the data scalability of human-centric pretraining methods by discarding depth information and exploring semantic information of RGB images in the frequency space by Discrete Cosine Transform (DCT). We further propose new annotation denoising auxiliary tasks with keypoints and DCT maps to enforce the RGB image extractor to learn fine-grained semantic information of human bodies. Our extensive experiments show that when pretrained on large-scale datasets (COCO and AIC datasets) without depth annotation, our model achieves better performance than state-of-the-art methods by +0.5 mAP on COCO, +1.4 PCKh on MPII and -0.51 EPE on Human3.6M for pose estimation, by +4.50 mIoU on Human3.6M for human parsing, by -3.14 MAE on SHA and -0.07 MAE on SHB for crowd counting, by +1.1 F1 score on SHA and +0.8 F1 score on SHA for crowd localization, and by +0.1 mAP on Market1501 and +0.8 mAP on MSMT for person ReID. We also validate the effectiveness of our method on MPII+NTURGBD datasets
Abstract (translated)
人本感知是多样化的计算机视觉任务的核心，并一直是长期的研究焦点。然而，先前的研究分别独立研究这些以人类为中心的任务，其性能很大程度上受限于公共特定任务数据集的大小。近期的人本方法利用额外的模态（如深度信息）来学习细粒度语义信息，但由于对视角敏感以及互联网上的RGB-D数据稀缺，这限制了预训练模型的好处。 本文通过放弃深度信息并在频域中探索RGB图像的语义信息来改进人本预训练方法的数据扩展性。我们进一步提出使用关键点和离散余弦变换（DCT）图的新注释去噪辅助任务，以促使RGB图像提取器学习人体细粒度语义信息。 我们的广泛实验表明，在大规模数据集（COCO和AIC数据集）上进行预训练且不依赖深度标注的情况下，我们所提出的方法在姿态估计、人体解析、人群计数、人群定位以及人员重识别等任务中均优于现有的最先进方法。例如，在COCO数据集上的mAP提高0.5%，MPII数据集的PCKh指标提高1.4%，Human3.6M数据集中EPE（端点误差）降低0.51；在人体解析方面，Human3.6M数据集的mIoU提高了4.50分。此外，在人群计数任务上，SHA和SHB数据集上的MAE分别降低了3.14和0.07。在人群定位任务中，SHA和SHB数据集中的F1分数增加了1.1和0.8。对于人员重识别，Market1501和MSMT数据集中mAP分别提高了0.1和0.8。 我们还通过MPII+NTURGBD数据集验证了该方法的有效性。
URL
https://arxiv.org/abs/2504.20800
