Abstract
Person Re-identification (ReID) systems identify individuals across images or video frames and play a critical role in various real-world applications. However, many ReID methods are influenced by sensitive attributes such as gender, pose, and body mass index (BMI), which vary in uncontrolled environments, leading to biases and reduced generalization. To address this, we extend the concept of expressivity to the body recognition domain to better understand how ReID models encode these attributes. Expressivity, defined as the mutual information between feature vector representations and specific attributes, is computed using a secondary neural network that takes feature and attribute vectors as inputs. This provides a quantitative framework for analyzing the extent to which sensitive attributes are embedded in the model's representations. We apply expressivity analysis to SemReID, a state-of-the-art self-supervised ReID model, and find that BMI consistently exhibits the highest expressivity scores in the model's final layers, underscoring its dominant role in feature encoding. In the final attention layer of the trained network, the expressivity order for body attributes is BMI > Pitch > Yaw > Gender, highlighting their relative importance in learned representations. Additionally, expressivity values evolve progressively across network layers and training epochs, reflecting a dynamic encoding of attributes during feature extraction. These insights emphasize the influence of body-related attributes on ReID models and provide a systematic methodology for identifying and mitigating attribute-driven biases. By leveraging expressivity analysis, we offer valuable tools to enhance the fairness, robustness, and generalization of ReID systems in diverse real-world settings.
Abstract (translated)
人员重识别（ReID）系统用于在图像或视频帧中识别个人，这对各种实际应用至关重要。然而，许多ReID方法会受到性别、姿势和体质指数（BMI）等敏感属性的影响，在不受控制的环境中这些属性变化较大，导致偏见并降低了模型的一般化能力。为了解决这个问题，我们将可表达性的概念扩展到人体识别领域，以更好地理解ReID模型如何编码这些属性。可表达性被定义为特征向量表示与特定属性之间的互信息，并通过一个辅助神经网络计算得出，该网络使用特征和属性向量作为输入。这提供了一个定量框架来分析敏感属性在模型表示中的嵌入程度。 我们将这种可表达性分析应用于SemReID，这是一种最先进的自我监督的ReID模型。我们发现，在模型最终层中，BMI始终表现出最高的可表达性得分，强调了它在特征编码中的主导作用。在网络训练后的最后注意力层，人体相关属性的重要性排序为：BMI > Pitch（俯仰角）> Yaw（偏航角）> 性别。 此外，随着网络层次的增加和训练周期的发展，这些可表达性的值也逐渐变化，反映了在特征提取过程中对属性编码动态调整的过程。这些见解强调了身体相关属性对ReID模型的影响，并提供了一种系统的方法来识别和缓解由属性驱动的偏见。 通过利用这种可表达性分析方法，我们为提高ReID系统在各种现实世界场景中的公平性、鲁棒性和泛化能力提供了有价值的工具。
URL
https://arxiv.org/abs/2503.06451
