Abstract
Crowd instance segmentation is a crucial task with a wide range of applications, including surveillance and transportation. Currently, point labels are common in crowd datasets, while region labels (e.g., boxes) are rare and inaccurate. The masks obtained through segmentation help to improve the accuracy of region labels and resolve the correspondence between individual location coordinates and crowd density maps. However, directly applying currently popular large foundation models such as SAM does not yield ideal results in dense crowds. To this end, we first propose Dense Point-to-Mask Optimization (DPMO), which integrates SAM with the Nearest Neighbor Exclusive Circle (NNEC) constraint to generate dense instance segmentation from point annotations. With DPMO and manual correction, we obtain mask annotations from the existing point annotations for traditional crowd datasets. Then, to predict instance segmentation in dense crowds, we propose a Reinforced Point Selection (RPS) framework trained with Group Relative Policy Optimization (GRPO), which selects the best predicted point from a sampling of the initial point prediction. Through extensive experiments, we achieve state-of-the-art crowd instance segmentation performance on ShanghaiTech, UCF-QNRF, JHU-CROWD++, and NWPU-Crowd datasets. Furthermore, we design new loss functions supervised by masks that boost counting performance across different models, demonstrating the significant role of mask annotations in enhancing counting accuracy.
Abstract (translated)
人群实例分割是一项具有广泛应用（包括监控和交通领域）的关键任务。目前，人群数据集中普遍采用点标注，而区域标注（如边界框）较为罕见且不够精确。通过分割获得的掩码有助于提升区域标注的准确性，并解决个体位置坐标与人群密度图之间的对应关系。然而，直接将当前流行的大型基础模型（如SAM）应用于密集人群场景时无法获得理想结果。为此，我们首先提出密集点-掩码优化方法（DPMO），该方法将SAM与最近邻互斥圆（NNEC）约束相结合，从点标注生成密集实例分割掩码。借助DPMO和人工修正，我们为传统人群数据集从现有点标注中获得了掩码标注。随后，为预测密集人群中的实例分割，我们提出了基于组相对策略优化（GRPO）训练的强化点选择（RPS）框架，该框架从初始点预测的采样结果中选择最优预测点。通过大量实验，我们在ShanghaiTech、UCF-QNRF、JHU-CROWD++和NWPU-Crowd数据集上达到了最先进的人群实例分割性能。此外，我们设计了由掩码监督的新损失函数，可提升不同模型的计数性能，证明了掩码标注在增强计数准确性方面的重要作用。
URL
https://arxiv.org/abs/2604.01742
