Re-identification
-
Trainee Action Recognition through Interaction Analysis in CCATT Mixed-Reality Training
2025-09-22 15:19:45Divya Mereddy, Marcos Quinones-Grueiro, Ashwin T S, Eduardo Davalos, Gautam Biswas, Kent Etherton, Tyler Davis, Katelyn Kay, Jill Lear, Benjamin GoldbergAbstract
This study examines how Critical Care Air Transport Team (CCATT) members are trained using mixed-reality simulations that replicate the high-pressure conditions of aeromedical evacuation. Each team - a physician, nurse, and respiratory therapist - must stabilize severely injured soldiers by managing ventilators, IV pumps, and suction devices during flight. Proficient performance requires clinical expertise and cognitive skills, such as situational awareness, rapid decision-making, effective communication, and coordinated task management, all of which must be maintained under stress. Recent advances in simulation and multimodal data analytics enable more objective and comprehensive performance evaluation. In contrast, traditional instructor-led assessments are subjective and may overlook critical events, thereby limiting generalizability and consistency. However, AI-based automated and more objective evaluation metrics still demand human input to train the AI algorithms to assess complex team dynamics in the presence of environmental noise and the need for accurate re-identification in multi-person tracking. To address these challenges, we introduce a systematic, data-driven assessment framework that combines Cognitive Task Analysis (CTA) with Multimodal Learning Analytics (MMLA). We have developed a domain-specific CTA model for CCATT training and a vision-based action recognition pipeline using a fine-tuned Human-Object Interaction model, the Cascade Disentangling Network (CDN), to detect and track trainee-equipment interactions over time. These interactions automatically yield performance indicators (e.g., reaction time, task duration), which are mapped onto a hierarchical CTA model tailored to CCATT operations, enabling interpretable, domain-relevant performance evaluations.
Abstract (translated)
本研究探讨了如何利用混合现实模拟训练危重病航空转运团队（CCATT）成员。每个团队包括一名医生、一名护士和一名呼吸治疗师，他们必须在飞行过程中通过管理呼吸机、静脉输液泵以及吸引装置来稳定严重受伤的士兵的生命体征。熟练的表现需要临床专业知识和认知技能，如情境意识、快速决策能力、有效沟通及任务协调管理，并且这些都需在压力环境下维持。 近期模拟技术与多模态数据分析的进步使得更客观全面地评估性能成为可能。相比之下，传统的由教师指导的评估方式较为主观，并可能忽略关键事件，从而限制了一致性和通用性。然而，基于人工智能的自动化和更客观的评价指标仍然需要人类输入来训练AI算法，以评估复杂团队动态中的环境噪音以及多人追踪时准确识别的需求。 为应对这些挑战，我们引入了一个系统性的、数据驱动的评估框架，该框架结合了认知任务分析（CTA）与多模态学习分析（MMLA）。我们开发了一个针对CCATT训练领域的特定CTA模型，并使用了一种基于视觉的动作识别管道——精调的人机交互模型，即级联解构网络（CDN），来检测并跟踪受训者与设备之间的互动。这些互动自动产生了性能指标（例如反应时间、任务持续时间等），它们映射到了一个层次化的CTA模型中，该模型专为CCATT操作设计，从而能够进行解释性且领域相关的绩效评估。
URL
https://arxiv.org/abs/2509.17888
PDF
-
SAMSON: 3rd Place Solution of LSVOS 2025 VOS Challenge
2025-09-22 08:30:34Yujie Xie, Hongyang Zhang, Zhihui Liu, Shihai RuanAbstract
Large-scale Video Object Segmentation (LSVOS) addresses the challenge of accurately tracking and segmenting objects in long video sequences, where difficulties stem from object reappearance, small-scale targets, heavy occlusions, and crowded scenes. Existing approaches predominantly adopt SAM2-based frameworks with various memory mechanisms for complex video mask generation. In this report, we proposed Segment Anything with Memory Strengthened Object Navigation (SAMSON), the 3rd place solution in the MOSE track of ICCV 2025, which integrates the strengths of stateof-the-art VOS models into an effective paradigm. To handle visually similar instances and long-term object disappearance in MOSE, we incorporate a long-term memorymodule for reliable object re-identification. Additionly, we adopt SAM2Long as a post-processing strategy to reduce error accumulation and enhance segmentation stability in long video sequences. Our method achieved a final performance of 0.8427 in terms of J &F in the test-set leaderboard.
Abstract (translated)
大规模视频目标分割（LSVOS）旨在解决在长时间视频序列中准确跟踪和分割对象的挑战，这些挑战包括对象再现、小规模目标、严重遮挡以及拥挤场景等问题。现有方法主要采用带有各种内存机制的SAM2框架来生成复杂视频掩码。在这份报告中，我们提出了一种名为“具有增强记忆功能的对象导航”（Segment Anything with Memory Strengthened Object Navigation, SAMSON）的方法，这是ICCV 2025 MOSE赛道的第三名解决方案。该方法将最先进的VOS模型的优势整合到一个有效的框架内。 为了处理MOSE中视觉相似实例和长时间对象消失的问题，我们引入了一个长期记忆模块来可靠地重新识别对象。此外，我们采用了SAM2Long作为后处理策略以减少误差累积，并增强在长视频序列中的分割稳定性。我们的方法在测试集排行榜上取得了J & F指标为0.8427的最终性能。 这个解决方案展示了如何利用先进的技术手段解决大规模和复杂的视频目标分割问题，同时通过创新的记忆机制解决了长期跟踪中出现的问题。
URL
https://arxiv.org/abs/2509.17500
PDF
-
Towards Anytime Retrieval: A Benchmark for Anytime Person Re-Identification
2025-09-20 11:20:22Xulin Li, Yan Lu, Bin Liu, Jiaze Li, Qinhong Yang, Tao Gong, Qi Chu, Mang Ye, Nenghai YuAbstract
In real applications, person re-identification (ReID) is expected to retrieve the target person at any time, including both daytime and nighttime, ranging from short-term to long-term. However, existing ReID tasks and datasets can not meet this requirement, as they are constrained by available time and only provide training and evaluation for specific scenarios. Therefore, we investigate a new task called Anytime Person Re-identification (AT-ReID), which aims to achieve effective retrieval in multiple scenarios based on variations in time. To address the AT-ReID problem, we collect the first large-scale dataset, AT-USTC, which contains 403k images of individuals wearing multiple clothes captured by RGB and IR cameras. Our data collection spans 21 months, and 270 volunteers were photographed on average 29.1 times across different dates or scenes, 4-15 times more than current datasets, providing conditions for follow-up investigations in AT-ReID. Further, to tackle the new challenge of multi-scenario retrieval, we propose a unified model named Uni-AT, which comprises a multi-scenario ReID (MS-ReID) framework for scenario-specific features learning, a Mixture-of-Attribute-Experts (MoAE) module to alleviate inter-scenario interference, and a Hierarchical Dynamic Weighting (HDW) strategy to ensure balanced training across all scenarios. Extensive experiments show that our model leads to satisfactory results and exhibits excellent generalization to all scenarios.
Abstract (translated)
在实际应用中，人员重识别（ReID）期望能够随时检索目标人物，包括白天和黑夜，并且可以涵盖短期到长期的时间跨度。然而，现有的人员重识别任务和数据集无法满足这一要求，因为它们受限于可用时间并仅针对特定场景提供训练和评估。因此，我们研究了一个新的任务，称为任何时间人员重识别（AT-ReID），目标是在基于时间变化的多场景中实现有效的检索。 为了解决AT-ReID问题，我们收集了首个大规模数据集AT-USTC，该数据集包含403,000张由RGB和IR相机拍摄的不同个体身着多种衣物的照片。我们的数据收集工作历时21个月，并且平均每个志愿者（共270人）在不同日期或场景中被拍摄了29.1次，这比目前的数据集多了4到15倍的拍摄次数，从而为后续关于AT-ReID的研究提供了条件。 此外，为了应对多场景检索带来的新挑战，我们提出了一种统一模型Uni-AT。该模型包含一个多场景重识别（MS-ReID）框架用于特定场景特征学习、一个属性专家混合体（MoAE）模块以减轻跨场景干扰以及一个分层动态加权（HDW）策略以确保所有场景的平衡训练。 广泛的实验表明，我们的模型表现出令人满意的结果，并且在所有场景中都展示了出色的泛化能力。
URL
https://arxiv.org/abs/2509.16635
PDF
-
A review of Recent Techniques for Person Re-Identification
2025-09-19 13:47:57Andrea Asperti, Salvatore Fiorilla, Simone Nardi, Lorenzo OrsiniAbstract
Person re-identification (ReId), a crucial task in surveillance, involves matching individuals across different camera views. The advent of Deep Learning, especially supervised techniques like Convolutional Neural Networks and Attention Mechanisms, has significantly enhanced person Re-ID. However, the success of supervised approaches hinges on vast amounts of annotated data, posing scalability challenges in data labeling and computational costs. To address these limitations, recent research has shifted towards unsupervised person re-identification. Leveraging abundant unlabeled data, unsupervised methods aim to overcome the need for pairwise labelled data. Although traditionally trailing behind supervised approaches, unsupervised techniques have shown promising developments in recent years, signalling a narrowing performance gap. Motivated by this evolving landscape, our survey pursues two primary objectives. First, we review and categorize significant publications in supervised person re-identification, providing an in-depth overview of the current state-of-the-art and emphasizing little room for further improvement in this domain. Second, we explore the latest advancements in unsupervised person re-identification over the past three years, offering insights into emerging trends and shedding light on the potential convergence of performance between supervised and unsupervised paradigms. This dual-focus survey aims to contribute to the evolving narrative of person re-identification, capturing both the mature landscape of supervised techniques and the promising outcomes in the realm of unsupervised learning.
Abstract (translated)
人员再识别（ReId）是监控领域的一项关键任务，它涉及在不同摄像头视角下匹配个体。随着深度学习的兴起，特别是卷积神经网络和注意力机制等监督技术的应用，人员再识别的效果得到了显著提升。然而，这些方法的成功很大程度上依赖于大量的标注数据，这对数据标签的可扩展性和计算成本提出了挑战。为了克服这些限制，最近的研究转向了无监督人员再识别领域。通过利用丰富的未标记数据，无监督方法旨在减少对成对标注数据的需求。尽管传统上落后于监督方法，近年来无监督技术已显示出令人鼓舞的发展，表明其与监督方法在性能上的差距正在缩小。 鉴于这一不断变化的格局，我们的综述追求两个主要目标。首先，我们回顾并分类了监督人员再识别领域的重大出版物，提供了当前最新技术水平的深入概述，并强调在这个领域进一步改进的空间有限。其次，我们探索了过去三年无监督人员再识别领域的最新进展，提供新兴趋势的见解，并阐明监督和无监督范式之间性能趋同的可能性。 这项双重焦点的综述旨在为不断发展的人员再识别叙事做出贡献，捕捉到监督技术成熟格局的同时，也关注无监督学习领域中充满希望的结果。
URL
https://arxiv.org/abs/2509.22690
PDF
-
SynBench: A Benchmark for Differentially Private Text Generation
2025-09-18 03:57:50Yidan Sun, Viktor Schlegel, Srinivasan Nandakumar, Iqra Zahid, Yuping Wu, Yulong Wu, Hao Li, Jie Zhang, Warren Del-Pinto, Goran Nenadic, Siew Kei Lam, Anil Anthony BharathAbstract
Data-driven decision support in high-stakes domains like healthcare and finance faces significant barriers to data sharing due to regulatory, institutional, and privacy concerns. While recent generative AI models, such as large language models, have shown impressive performance in open-domain tasks, their adoption in sensitive environments remains limited by unpredictable behaviors and insufficient privacy-preserving datasets for benchmarking. Existing anonymization methods are often inadequate, especially for unstructured text, as redaction and masking can still allow re-identification. Differential Privacy (DP) offers a principled alternative, enabling the generation of synthetic data with formal privacy assurances. In this work, we address these challenges through three key contributions. First, we introduce a comprehensive evaluation framework with standardized utility and fidelity metrics, encompassing nine curated datasets that capture domain-specific complexities such as technical jargon, long-context dependencies, and specialized document structures. Second, we conduct a large-scale empirical study benchmarking state-of-the-art DP text generation methods and LLMs of varying sizes and different fine-tuning strategies, revealing that high-quality domain-specific synthetic data generation under DP constraints remains an unsolved challenge, with performance degrading as domain complexity increases. Third, we develop a membership inference attack (MIA) methodology tailored for synthetic text, providing first empirical evidence that the use of public datasets - potentially present in pre-training corpora - can invalidate claimed privacy guarantees. Our findings underscore the urgent need for rigorous privacy auditing and highlight persistent gaps between open-domain and specialist evaluations, informing responsible deployment of generative AI in privacy-sensitive, high-stakes settings.
Abstract (translated)
在高风险领域（如医疗和金融）中，基于数据的决策支持面临着由于监管、机构和隐私问题而导致的数据共享障碍。尽管最近的生成式人工智能模型（例如大规模语言模型）在开放域任务中表现出色，但它们在敏感环境中的应用仍受限于不可预测的行为以及缺乏用于基准测试的隐私保护数据集。现有的匿名化方法通常不足以处理非结构化的文本，因为即使删除和屏蔽也可能允许重新识别信息。差分隐私（DP）提供了一种有原则的替代方案，它能够生成具有正式隐私保证的合成数据。 在本研究中，我们通过三项关键贡献来应对这些挑战： 1. 我们引入了一个全面的评估框架，包括标准化的实用性和保真度指标，并涵盖了九个经过策划的数据集，捕捉特定领域的复杂性，如技术术语、长上下文依赖和专门化的文档结构。 2. 我们进行了一项大规模实证研究，对最先进的差分隐私文本生成方法以及不同规模和微调策略的语言模型进行了基准测试。结果表明，在差分隐私约束下生成高质量的领域特定合成数据仍然是一个未解决的问题，并且随着领域复杂性的增加，性能会下降。 3. 我们开发了一种专门针对合成文本的成员推断攻击（MIA）方法，提供了首次实验证据，证明使用公共数据集——可能存在于预训练语料库中——可以使声称的隐私保证无效。 我们的发现强调了对生成式人工智能在隐私敏感、高风险环境中的负责任部署进行严格隐私审核的紧迫需求，并且指出了开放领域与专业评估之间的持久差距。
URL
https://arxiv.org/abs/2509.14594
PDF
-
Measuring Soft Biometric Leakage in Speaker De-Identification Systems
2025-09-17 22:52:01Seungmin Seo, Oleg Aulov, P. Jonathon PhillipsAbstract
We use the term re-identification to refer to the process of recovering the original speaker's identity from anonymized speech outputs. Speaker de-identification systems aim to reduce the risk of re-identification, but most evaluations focus only on individual-level measures and overlook broader risks from soft biometric leakage. We introduce the Soft Biometric Leakage Score (SBLS), a unified method that quantifies resistance to zero-shot inference attacks on non-unique traits such as channel type, age range, dialect, sex of the speaker, or speaking style. SBLS integrates three elements: direct attribute inference using pre-trained classifiers, linkage detection via mutual information analysis, and subgroup robustness across intersecting attributes. Applying SBLS with publicly available classifiers, we show that all five evaluated de-identification systems exhibit significant vulnerabilities. Our results indicate that adversaries using only pre-trained models - without access to original speech or system details - can still reliably recover soft biometric information from anonymized output, exposing fundamental weaknesses that standard distributional metrics fail to capture.
Abstract (translated)
我们使用“重新识别”一词来指代从匿名语音输出中恢复原始说话人身份的过程。说话人去标识化系统旨在降低重新识别的风险，但大多数评估仅关注个体层面的措施，并忽略了来自软生物特征泄漏的更广泛风险。为此，我们引入了软生物特征泄漏评分（SBLS），这是一种统一的方法，用于量化对零样本推理攻击的抗性，这种攻击针对的是非唯一特质，如通道类型、年龄范围、方言、说话人性别或讲话风格。 SBLS整合了三个要素：直接属性推断使用预训练分类器；通过互信息分析进行链接检测；以及跨交叉属性的子群稳健性。利用公开可用的分类器应用SBLS，我们展示了所有五个评估的去标识化系统都存在显著漏洞。我们的结果表明，即使对手仅使用预训练模型而无需访问原始语音或系统细节，仍然可以可靠地从匿名输出中恢复软生物特征信息，揭示了标准分布度量未能捕捉到的根本弱点。
URL
https://arxiv.org/abs/2509.14469
PDF
-
VCBench: Benchmarking LLMs in Venture Capital
2025-09-17 21:56:48Rick Chen, Joseph Ternasky, Afriyie Samuel Kwesi, Ben Griffin, Aaron Ontoyin Yin, Zakari Salifu, Kelvin Amoaba, Xianling Mu, Fuat Alican, Yigit IhlamurAbstract
Benchmarks such as SWE-bench and ARC-AGI demonstrate how shared datasets accelerate progress toward artificial general intelligence (AGI). We introduce VCBench, the first benchmark for predicting founder success in venture capital (VC), a domain where signals are sparse, outcomes are uncertain, and even top investors perform modestly. At inception, the market index achieves a precision of 1.9%. Y Combinator outperforms the index by a factor of 1.7x, while tier-1 firms are 2.9x better. VCBench provides 9,000 anonymized founder profiles, standardized to preserve predictive features while resisting identity leakage, with adversarial tests showing more than 90% reduction in re-identification risk. We evaluate nine state-of-the-art large language models (LLMs). DeepSeek-V3 delivers over six times the baseline precision, GPT-4o achieves the highest F0.5, and most models surpass human benchmarks. Designed as a public and evolving resource available at this http URL, VCBench establishes a community-driven standard for reproducible and privacy-preserving evaluation of AGI in early-stage venture forecasting.
Abstract (translated)
诸如SWE-bench和ARC-AGI这样的基准测试表明，共享数据集如何加速向通用人工智能（AGI）的进展。我们介绍了VCBench，这是首个用于预测创业投资（VC）领域创始人成功的基准测试，在该领域中信号稀疏、结果不确定，并且即使是顶级投资者的表现也较为平庸。在初始阶段，市场指数达到了1.9%的精度。而Y Combinator比市场指数高出1.7倍，一级公司则高出2.9倍。VCBench提供了9,000个匿名化的创始人资料，这些资料被标准化以保留预测特征的同时又能防止身份泄露，对抗性测试显示重新识别的风险减少了超过90%。 我们评估了九种最先进的大型语言模型（LLM）。DeepSeek-V3的精度比基准高出六倍以上，GPT-4o实现了最高的F0.5得分，大多数模型都超过了人类基准。VCBench作为一个公开且不断更新的资源，在此网址可获取，它为早期阶段的风险投资预测中AGI的可重复性和隐私保护评估建立了社区驱动的标准。
URL
https://arxiv.org/abs/2509.14448
PDF
-
DS@GT AnimalCLEF: Triplet Learning over ViT Manifolds with Nearest Neighbor Classification for Animal Re-identification
2025-09-15 18:31:01Anthony Miyaguchi, Chandrasekaran Maruthaiyannan, Charles R. ClarkAbstract
This paper details the DS@GT team's entry for the AnimalCLEF 2025 re-identification challenge. Our key finding is that the effectiveness of post-hoc metric learning is highly contingent on the initial quality and domain-specificity of the backbone embeddings. We compare a general-purpose model (DINOv2) with a domain-specific model (MegaDescriptor) as a backbone. A K-Nearest Neighbor classifier with robust thresholding then identifies known individuals or flags new ones. While a triplet-learning projection head improved the performance of the specialized MegaDescriptor model by 0.13 points, it yielded minimal gains (0.03) for the general-purpose DINOv2 on averaged BAKS and BAUS. We demonstrate that the general-purpose manifold is more difficult to reshape for fine-grained tasks, as evidenced by stagnant validation loss and qualitative visualizations. This work highlights the critical limitations of refining general-purpose features for specialized, limited-data re-ID tasks and underscores the importance of domain-specific pre-training. The implementation for this work is publicly available at this http URL.
Abstract (translated)
本文详细介绍了DS@GT团队在AnimalCLEF 2025再识别挑战中的参赛方案。我们的主要发现是，后处理度量学习的有效性高度依赖于基础嵌入的初始质量和领域特定性。我们比较了一般用途模型（DINOv2）和专门领域的模型（MegaDescriptor）作为骨干网络的表现。随后使用K-近邻分类器并结合鲁棒阈值来识别已知个体或标记新的个体。虽然三元组学习投影头使专业的MegaDescriptor模型的性能提高了0.13分，但对于一般用途的DINOv2模型，在平均BAKS和BAUS上的改进微乎其微（仅为0.03）。我们展示了通用流形在细粒度任务中更难以重塑，这一点从验证损失停滞以及定性可视化中得以体现。这项工作突显了为专门且数据有限的再识别任务优化通用特征所面临的重要限制，并强调了领域特定预训练的重要性。该工作的实现代码已公开发布于[此处提供URL]。
URL
https://arxiv.org/abs/2509.12353
PDF
-
Hierarchical Identity Learning for Unsupervised Visible-Infrared Person Re-Identification
2025-09-15 05:10:43Haonan Shi, Yubin Wang, De Cheng, Lingfeng He, Nannan Wang, Xinbo GaoAbstract
Unsupervised visible-infrared person re-identification (USVI-ReID) aims to learn modality-invariant image features from unlabeled cross-modal person datasets by reducing the modality gap while minimizing reliance on costly manual annotations. Existing methods typically address USVI-ReID using cluster-based contrastive learning, which represents a person by a single cluster center. However, they primarily focus on the commonality of images within each cluster while neglecting the finer-grained differences among them. To address the limitation, we propose a Hierarchical Identity Learning (HIL) framework. Since each cluster may contain several smaller sub-clusters that reflect fine-grained variations among images, we generate multiple memories for each existing coarse-grained cluster via a secondary clustering. Additionally, we propose Multi-Center Contrastive Learning (MCCL) to refine representations for enhancing intra-modal clustering and minimizing cross-modal discrepancies. To further improve cross-modal matching quality, we design a Bidirectional Reverse Selection Transmission (BRST) mechanism, which establishes reliable cross-modal correspondences by performing bidirectional matching of pseudo-labels. Extensive experiments conducted on the SYSU-MM01 and RegDB datasets demonstrate that the proposed method outperforms existing approaches. The source code is available at: this https URL.
Abstract (translated)
无监督可见光-红外行人重识别（USVI-ReID）的目标是从未标注的跨模态行人数据集中学习模态不变性图像特征，同时减少模态差距并尽量减少对成本高昂的人工注释的依赖。现有方法通常通过基于聚类的对比学习来解决 USVI-ReID 问题，这种方法用单个聚类中心表示一个人。然而，它们主要关注每个聚类内图像之间的共性而忽略了这些图像之间细微的区别。为了解决这一局限性，我们提出了一种层次身份学习（HIL）框架。 由于每个大聚类可能包含反映图间细微差异的多个较小子聚类，我们通过二次聚类为每个现有粗粒度聚类生成多个记忆库。此外，我们提出了多中心对比学习（MCCL），以细化表示方法，增强模内聚类并最小化跨模式差距。为了进一步提高跨模态匹配质量，我们设计了一种双向逆向选择传输（BRST）机制，通过进行伪标签的双向匹配来建立可靠的跨模态对应关系。 在 SYSU-MM01 和 RegDB 数据集上进行的大量实验表明，所提出的方法优于现有方法。源代码可在以下网址获取：此 https URL。
URL
https://arxiv.org/abs/2509.11587
PDF
-
Contextualized Multimodal Lifelong Person Re-Identification in Hybrid Clothing States
2025-09-14 12:46:39Robert Long, Rongxin Jiang, Mingrui YanAbstract
Person Re-Identification (ReID) has several challenges in real-world surveillance systems due to clothing changes (CCReID) and the need for maintaining continual learning (LReID). Previous existing methods either develop models specifically for one application, which is mostly a same-cloth (SC) setting or treat CCReID as its own separate sub-problem. In this work, we will introduce the LReID-Hybrid task with the goal of developing a model to achieve both SC and CC while learning in a continual setting. Mismatched representations and forgetting from one task to the next are significant issues, we address this with CMLReID, a CLIP-based framework composed of two novel tasks: (1) Context-Aware Semantic Prompt (CASP) that generates adaptive prompts, and also incorporates context to align richly multi-grained visual cues with semantic text space; and (2) Adaptive Knowledge Fusion and Projection (AKFP) which produces robust SC/CC prototypes through the use of a dual-path learner that aligns features with our Clothing-State-Aware Projection Loss. Experiments performed on a wide range of datasets and illustrate that CMLReID outperforms all state-of-the-art methods with strong robustness and generalization despite clothing variations and a sophisticated process of sequential learning.
Abstract (translated)
人员再识别（Re-ID）在实际的监控系统中面临若干挑战，包括因穿着变化（CCReID）和持续学习的需求（LReID）。以往的方法要么开发专门针对单一应用场景的模型，这通常是基于相同衣物设置的情况；要么将穿着变化视为一个独立的问题来处理。在这项工作中，我们将介绍LReID-Hybrid任务，旨在开发一种能够在连续学习环境中同时适应相同衣物与不同衣物情景的模型。 在从一项任务过渡到另一项任务时出现表示不匹配和遗忘问题是很重要的挑战，我们通过CMLReID（基于CLIP框架的方法）来解决这一问题。该方法由两个新颖的任务组成：(1) 具有上下文感知语义提示（Context-Aware Semantic Prompt, CASP）的任务，它可以生成自适应的提示，并将上下文信息纳入其中以对齐多粒度视觉线索与语义文本空间；以及 (2) 通过使用双路径学习器并结合衣物状态感知投影损失来产生稳健相同/不同衣物原型的自适应知识融合和投影（Adaptive Knowledge Fusion and Projection, AKFP）任务。实验在广泛的数据集上进行，结果表明CMLReID方法不仅超越了所有现有的最先进的技术，在面对服装变化的情况下还能保持强大的鲁棒性和泛化能力，并且即使是在复杂的序列学习过程中也是如此。
URL
https://arxiv.org/abs/2509.11247
PDF
-
An HMM-based framework for identity-aware long-term multi-object tracking from sparse and uncertain identification: use case on long-term tracking in livestock
2025-09-12 04:39:38Anne Marthe Sophie Ngo Bibinbe, Chiron Bang, Patrick Gagnon, Jamie Ahloy-Dallaire, Eric R. PaquetAbstract
The need for long-term multi-object tracking (MOT) is growing due to the demand for analyzing individual behaviors in videos that span several minutes. Unfortunately, due to identity switches between objects, the tracking performance of existing MOT approaches decreases over time, making them difficult to apply for long-term tracking. However, in many real-world applications, such as in the livestock sector, it is possible to obtain sporadic identifications for some of the animals from sources like feeders. To address the challenges of long-term MOT, we propose a new framework that combines both uncertain identities and tracking using a Hidden Markov Model (HMM) formulation. In addition to providing real-world identities to animals, our HMM framework improves the F1 score of ByteTrack, a leading MOT approach even with re-identification, on a 10 minute pig tracking dataset with 21 identifications at the pen's feeding station. We also show that our approach is robust to the uncertainty of identifications, with performance increasing as identities are provided more frequently. The improved performance of our HMM framework was also validated on the MOT17 and MOT20 benchmark datasets using both ByteTrack and FairMOT. The code for this new HMM framework and the new 10-minute pig tracking video dataset are available at: this https URL
Abstract (translated)
由于对分析跨越几分钟视频中个体行为的需求增加，长期多目标跟踪（MOT）的重要性正在上升。不幸的是，现有的MOT方法由于对象之间的身份切换，在长时间内跟踪性能会下降，这使得它们难以用于长时跟踪。然而，在许多实际应用中，例如在畜牧业领域，可以通过诸如喂食器等来源间歇性地识别一些动物的身份。为了解决长期MOT的挑战，我们提出了一种新的框架，该框架结合了不确定身份和追踪，并利用隐马尔可夫模型（HMM）的形式化方法。 除了提供现实世界中的动物身份外，我们的HMM框架还能在10分钟猪跟踪数据集上提升ByteTrack这一领先MOT方法的F1得分，即使是在有重识别的情况下也有效。该数据集中包含21个在围栏喂食站处的身份识别信息。我们还展示了我们的方法对于身份识别不确定性的鲁棒性，并且随着提供的身份信息频率增加，性能也会相应提高。 我们在MOT17和MOT20基准测试数据集上使用ByteTrack和FairMOT验证了我们HMM框架的改进效果。此新HMM框架以及新的10分钟猪跟踪视频数据集的相关代码可以在以下网址获取：[此处插入URL]
URL
https://arxiv.org/abs/2509.09962
PDF
-
Similarity-based Outlier Detection for Noisy Object Re-Identification Using Beta Mixtures
2025-09-10 18:42:19Waqar Ahmad, Evan Murphy, Vladimir A. KrylovAbstract
Object re-identification (Re-ID) methods are highly sensitive to label noise, which typically leads to significant performance degradation. We address this challenge by reframing Re-ID as a supervised image similarity task and adopting a Siamese network architecture trained to capture discriminative pairwise relationships. Central to our approach is a novel statistical outlier detection (OD) framework, termed Beta-SOD (Beta mixture Similarity-based Outlier Detection), which models the distribution of cosine similarities between embedding pairs using a two-component Beta distribution mixture model. We establish a novel identifiability result for mixtures of two Beta distributions, ensuring that our learning task is this http URL proposed OD step complements the Re-ID architecture combining binary cross-entropy, contrastive, and cosine embedding losses that jointly optimize feature-level similarity this http URL demonstrate the effectiveness of Beta-SOD in de-noising and Re-ID tasks for person Re-ID, on CUHK03 and Market-1501 datasets, and vehicle Re-ID, on VeRi-776 dataset. Our method shows superior performance compared to the state-of-the-art methods across various noise levels (10-30\%), demonstrating both robustness and broad applicability in noisy Re-ID scenarios. The implementation of Beta-SOD is available at: this https URL
Abstract (translated)
对象重识别（Re-ID）方法对标签噪声非常敏感，通常会导致性能显著下降。我们通过将Re-ID重构为监督下的图像相似度任务，并采用经过训练以捕捉判别性成对关系的Siamese网络架构来应对这一挑战。我们的方法核心是一种新的统计异常值检测（OD）框架，称为Beta-SOD（基于余弦相似性的贝塔混合异常值检测），该框架使用双组分贝塔分布混合模型来建模嵌入对之间余弦相似度的分布。 我们为两个贝塔分布的混合物建立了新颖的可识别性结果，确保了我们的学习任务具有良好的理论基础。所提出的OD步骤与结合二元交叉熵、对比损失和余弦嵌入损失的Re-ID架构相补充，共同优化特征级相似度。我们在CUHK03和Market-1501数据集上展示了Beta-SOD在人员重识别去噪和Re-ID任务中的有效性，并在VeRi-776数据集上的车辆Re-ID中也展示了相同的效果。 我们的方法在各种噪声水平（10%-30%）下均优于当前最先进的方法，显示了其在嘈杂的Re-ID场景下的鲁棒性和广泛应用性。Beta-SOD的实现可以在以下网址获取：[请在此处插入实际链接]
URL
https://arxiv.org/abs/2509.08926
PDF
-
A Re-ranking Method using K-nearest Weighted Fusion for Person Re-identification
2025-09-04 09:29:25Quang-Huy Che, Le-Chuong Nguyen, Gia-Nghia Tran, Dinh-Duy Phan, Vinh-Tiep NguyenAbstract
In person re-identification, re-ranking is a crucial step to enhance the overall accuracy by refining the initial ranking of retrieved results. Previous studies have mainly focused on features from single-view images, which can cause view bias and issues like pose variation, viewpoint changes, and occlusions. Using multi-view features to present a person can help reduce view bias. In this work, we present an efficient re-ranking method that generates multi-view features by aggregating neighbors' features using K-nearest Weighted Fusion (KWF) method. Specifically, we hypothesize that features extracted from re-identification models are highly similar when representing the same identity. Thus, we select K neighboring features in an unsupervised manner to generate multi-view features. Additionally, this study explores the weight selection strategies during feature aggregation, allowing us to identify an effective strategy. Our re-ranking approach does not require model fine-tuning or extra annotations, making it applicable to large-scale datasets. We evaluate our method on the person re-identification datasets Market1501, MSMT17, and Occluded-DukeMTMC. The results show that our method significantly improves Rank@1 and mAP when re-ranking the top M candidates from the initial ranking results. Specifically, compared to the initial results, our re-ranking method achieves improvements of 9.8%/22.0% in Rank@1 on the challenging datasets: MSMT17 and Occluded-DukeMTMC, respectively. Furthermore, our approach demonstrates substantial enhancements in computational efficiency compared to other re-ranking methods.
Abstract (translated)
在人员再识别中，重排名是一个关键步骤，通过优化检索结果的初始排序来提高整体准确性。以往的研究主要集中在单一视角图像特征上，这可能导致视角偏差以及姿态变化、视点改变和遮挡等问题。利用多视角特征表示一个人有助于减少视角偏差。在这项工作中，我们提出了一种高效的重排名方法，该方法通过使用K最近邻加权融合(KWF)法聚合邻居的特征来生成多视角特征。具体而言，我们假设从再识别模型中提取的同一种身份的特征高度相似。因此，我们在无监督的方式下选择K个相邻的特征以生成多视角特征。此外，这项研究还探讨了在特征聚合过程中权重的选择策略，使我们可以确定有效的策略。我们的重排名方法不需要对模型进行微调或添加额外注释，使其适用于大规模数据集。我们在Market1501、MSMT17和Occluded-DukeMTMC等人员再识别数据集中评估了该方法。结果表明，在初始排序的前M名候选人中重新排列时，我们的方法在Rank@1和mAP上显著提升。具体来说，与初始结果相比，我们提出的重排名方法分别在具有挑战性的MSMT17和Occluded-DukeMTMC数据集上实现了9.8%和22.0%的Rank@1改进。此外，我们的方法展示了比其他重排名方法更高的计算效率。
URL
https://arxiv.org/abs/2509.04050
PDF
-
A Comprehensive Guide to Differential Privacy: From Theory to User Expectations
2025-09-03 13:23:10Napsu Karmitsa, Antti Airola, Tapio Pahikkala, Tinja Pitk\"am\"akiAbstract
The increasing availability of personal data has enabled significant advances in fields such as machine learning, healthcare, and cybersecurity. However, this data abundance also raises serious privacy concerns, especially in light of powerful re-identification attacks and growing legal and ethical demands for responsible data use. Differential privacy (DP) has emerged as a principled, mathematically grounded framework for mitigating these risks. This review provides a comprehensive survey of DP, covering its theoretical foundations, practical mechanisms, and real-world applications. It explores key algorithmic tools and domain-specific challenges - particularly in privacy-preserving machine learning and synthetic data generation. The report also highlights usability issues and the need for improved communication and transparency in DP systems. Overall, the goal is to support informed adoption of DP by researchers and practitioners navigating the evolving landscape of data privacy.
Abstract (translated)
个人数据的日益普及在机器学习、医疗保健和网络安全等领域取得了显著进展。然而，这种数据丰富性也引发了严重的隐私问题，尤其是在强大的重新识别攻击以及对负责任的数据使用日益增长的法律和伦理要求背景下。差分隐私（DP）作为一种原则性和数学基础坚实的框架已出现，旨在缓解这些风险。这篇综述全面介绍了差分隐私，涵盖了其理论基础、实际机制及现实世界的应用。它探讨了关键算法工具和特定领域的挑战——特别是隐私保护机器学习和合成数据生成方面的挑战。报告还强调了使用性问题以及改进差分隐私系统中沟通与透明度的需求。总体而言，目标是支持研究人员和从业者在不断变化的数据隐私领域内做出明智的差分隐私采用决策。
URL
https://arxiv.org/abs/2509.03294
PDF
-
Background Matters Too: A Language-Enhanced Adversarial Framework for Person Re-Identification
2025-09-03 05:38:22Kaicong Huang, Talha Azfar, Jack M. Reilly, Thomas Guggisberg, Ruimin KeAbstract
Person re-identification faces two core challenges: precisely locating the foreground target while suppressing background noise and extracting fine-grained features from the target region. Numerous visual-only approaches address these issues by partitioning an image and applying attention modules, yet they rely on costly manual annotations and struggle with complex occlusions. Recent multimodal methods, motivated by CLIP, introduce semantic cues to guide visual understanding. However, they focus solely on foreground information, but overlook the potential value of background cues. Inspired by human perception, we argue that background semantics are as important as the foreground semantics in ReID, as humans tend to eliminate background distractions while focusing on target appearance. Therefore, this paper proposes an end-to-end framework that jointly models foreground and background information within a dual-branch cross-modal feature extraction pipeline. To help the network distinguish between the two domains, we propose an intra-semantic alignment and inter-semantic adversarial learning strategy. Specifically, we align visual and textual features that share the same semantics across domains, while simultaneously penalizing similarity between foreground and background features to enhance the network's discriminative power. This strategy drives the model to actively suppress noisy background regions and enhance attention toward identity-relevant foreground cues. Comprehensive experiments on two holistic and two occluded ReID benchmarks demonstrate the effectiveness and generality of the proposed method, with results that match or surpass those of current state-of-the-art approaches.
Abstract (translated)
人员再识别面临两大核心挑战：一是精确定位前景目标同时抑制背景噪声，二是从目标区域提取细微特征。众多仅依赖视觉的方法通过图像分割和应用注意力模块来解决这些问题，但它们依赖于昂贵的手动注释，并且在处理复杂遮挡时表现不佳。受CLIP方法启发的最近多模态方法引入了语义线索以指导视觉理解，然而这些方法仅仅关注前景信息，而忽略了背景线索的潜在价值。 受到人类感知机制的启发，我们认为，在再识别任务中，背景语义与前景语义同样重要。这是因为人在关注目标外观时往往会消除背景干扰。因此，本文提出了一种端到端框架，该框架在双分支跨模态特征提取管道内共同建模前景和背景信息。为了帮助网络区分这两个领域，我们提出了一种内部语义对齐和跨域语义对抗学习策略：具体而言，在不同领域的视觉与文本特征共享相同语义时进行对齐，同时惩罚前景与背景特征之间的相似度以增强网络的判别能力。这种策略促使模型积极抑制噪声背景区域并提高对身份相关的前景线索的关注。 在两个全景和两个遮挡再识别基准上的全面实验表明了所提出方法的有效性和通用性，并且其结果可匹敌或超越目前最先进的方法。
URL
https://arxiv.org/abs/2509.03032
PDF
-
Generalizable Object Re-Identification via Visual In-Context Prompting
2025-08-28 21:24:06Zhizhong Huang, Xiaoming LiuAbstract
Current object re-identification (ReID) methods train domain-specific models (e.g., for persons or vehicles), which lack generalization and demand costly labeled data for new categories. While self-supervised learning reduces annotation needs by learning instance-wise invariance, it struggles to capture \textit{identity-sensitive} features critical for ReID. This paper proposes Visual In-Context Prompting~(VICP), a novel framework where models trained on seen categories can directly generalize to unseen novel categories using only \textit{in-context examples} as prompts, without requiring parameter adaptation. VICP synergizes LLMs and vision foundation models~(VFM): LLMs infer semantic identity rules from few-shot positive/negative pairs through task-specific prompting, which then guides a VFM (\eg, DINO) to extract ID-discriminative features via \textit{dynamic visual prompts}. By aligning LLM-derived semantic concepts with the VFM's pre-trained prior, VICP enables generalization to novel categories, eliminating the need for dataset-specific retraining. To support evaluation, we introduce ShopID10K, a dataset of 10K object instances from e-commerce platforms, featuring multi-view images and cross-domain testing. Experiments on ShopID10K and diverse ReID benchmarks demonstrate that VICP outperforms baselines by a clear margin on unseen categories. Code is available at this https URL.
Abstract (translated)
当前的对象重识别（ReID）方法训练特定领域的模型（例如针对人员或车辆），这限制了它们的泛化能力，并且对于新类别需要昂贵的标注数据。虽然自我监督学习通过实例级别的不变性学习减少了注释需求，但它难以捕捉对重识别至关重要的“身份敏感”特征。本文提出了视觉上下文提示（VICP）框架，这是一种新型方法，在该框架中，训练好的模型可以直接使用仅有的“在上下文中的示例”作为提示来泛化到未见过的新类别，而无需调整参数。VICP将大型语言模型（LLM）和视觉基础模型（VFM）相结合：LLMs 通过特定任务的提示从少量正负对中推断出语义身份规则，然后指导 VFM（例如 DINO）提取识别特征，借助于“动态视觉提示”。通过将 LLM 推导出来的语义概念与 VFM 预训练的基础模型对齐，VICP 实现了向新类别的泛化，并消除了特定数据集重新训练的需要。为了支持评估，我们引入了一个名为 ShopID10K 的数据集，该数据集包含来自电子商务平台的 10,000 多个对象实例，具有多视角图像和跨域测试功能。在 ShopID10K 和各种 ReID 基准上的实验表明，VICP 在未见过的类别上明显优于基线方法。代码可在以下 URL 获得：[此 https URL]。
URL
https://arxiv.org/abs/2508.21222
PDF
-
FedKLPR: Personalized Federated Learning for Person Re-Identification with Adaptive Pruning
2025-08-24 16:11:41Po-Hsien Yu, Yu-Syuan Tseng, Shao-Yi ChienAbstract
Person re-identification (Re-ID) is a fundamental task in intelligent surveillance and public safety. Federated learning (FL) offers a privacy-preserving solution by enabling collaborative model training without centralized data collection. However, applying FL to real-world re-ID systems faces two major challenges: statistical heterogeneity across clients due to non-IID data distributions, and substantial communication overhead caused by frequent transmission of large-scale models. To address these issues, we propose FedKLPR, a lightweight and communication-efficient federated learning framework for person re-identification. FedKLPR introduces four key components. First, the KL-Divergence Regularization Loss (KLL) constrains local models by minimizing the divergence from the global feature distribution, effectively mitigating the effects of statistical heterogeneity and improving convergence stability under non-IID conditions. Secondly, KL-Divergence-Prune Weighted Aggregation (KLPWA) integrates pruning ratio and distributional similarity into the aggregation process, thereby improving the robustness of the global model while significantly reducing communication overhead. Furthermore, sparse Activation Skipping (SAS) mitigates the dilution of critical parameters during the aggregation of pruned client models by excluding zero-valued weights from the update process. Finally, Cross-Round Recovery (CRR) introduces a dynamic pruning control mechanism that halts pruning when necessary, enabling deeper compression while maintaining model accuracy. Experimental results on eight benchmark datasets demonstrate that FedKLPR achieves significant communication reduction. Compared with the state-of-the-art, FedKLPR reduces 33\%-38\% communication cost on ResNet-50 and 20\%-40\% communication cost on ResNet-34, while maintaining model accuracy within 1\% degradation.
Abstract (translated)
人员重新识别（Re-ID）是智能监控和公共安全中的一个基本任务。联邦学习（FL）通过允许在不进行集中式数据收集的情况下协作模型训练，提供了一种保护隐私的解决方案。然而，在实际的Re-ID系统中应用FL面临着两个主要挑战：由于非独立同分布（non-IID）的数据分布导致客户端之间的统计异质性以及由大规模模型频繁传输引起的大量通信开销。 为了解决这些问题，我们提出了FedKLPR，这是一种轻量级且通信效率高的联邦学习框架，用于人员重新识别。FedKLPR引入了四个关键组件： 1. **KL散度正则化损失（KLL）**：通过最小化从全局特征分布的发散来约束本地模型，有效地减轻统计异质性的影响，并在非独立同分布条件下提高收敛稳定性。 2. **KL散度剪枝加权聚合（KLPWA）**：将剪枝比率和分布相似性集成到聚合过程中，从而提高了全局模型的鲁棒性，并显著减少了通信开销。 3. **稀疏激活跳过（SAS）**：通过在更新过程中排除零值权重，减轻了在合并经过修剪的客户端模型时关键参数被稀释的问题。 4. **跨轮次恢复（CRR）**：引入了一种动态剪枝控制机制，在必要时停止剪枝操作，使更深层次的压缩成为可能，同时保持模型准确性。 实验结果表明，在八个基准数据集上，FedKLPR实现了显著的通信量减少。与现有最佳方法相比，FedKLPR在ResNet-50上的通信成本降低了33%-38%，在ResNet-34上则减少了20%-40%的同时，保持了模型精度不超过1%的退化。
URL
https://arxiv.org/abs/2508.17431
PDF
-
FastTracker: Real-Time and Accurate Visual Tracking
2025-08-20 02:59:53Hamidreza Hashempoor, Yu Dong HwangAbstract
Conventional multi-object tracking (MOT) systems are predominantly designed for pedestrian tracking and often exhibit limited generalization to other object categories. This paper presents a generalized tracking framework capable of handling multiple object types, with a particular emphasis on vehicle tracking in complex traffic scenes. The proposed method incorporates two key components: (1) an occlusion-aware re-identification mechanism that enhances identity preservation for heavily occluded objects, and (2) a road-structure-aware tracklet refinement strategy that utilizes semantic scene priors such as lane directions, crosswalks, and road boundaries to improve trajectory continuity and accuracy. In addition, we introduce a new benchmark dataset comprising diverse vehicle classes with frame-level tracking annotations, specifically curated to support evaluation of vehicle-focused tracking methods. Extensive experimental results demonstrate that the proposed approach achieves robust performance on both the newly introduced dataset and several public benchmarks, highlighting its effectiveness in general-purpose object tracking. While our framework is designed for generalized multi-class tracking, it also achieves strong performance on conventional benchmarks, with HOTA scores of 66.4 on MOT17 and 65.7 on MOT20 test sets. Code and Benchmark are available: this http URL, this http URL.
Abstract (translated)
传统的多目标跟踪（MOT）系统主要用于行人追踪，往往在处理其他对象类别时表现出有限的泛化能力。本文提出了一种通用化的跟踪框架，能够应对多种物体类型，并特别强调了在复杂交通场景中进行车辆跟踪的能力。该方法包含两个关键组件：(1) 一种遮挡感知重识别机制，增强了对严重被遮挡物体的身份保持；(2) 一种道路结构感知的轨迹细化策略，利用语义场景先验信息（如车道方向、人行横道和道路边界）来提高轨迹连续性和准确性。此外，我们还引入了一个新的基准数据集，该数据集包含多种车辆类别，并带有逐帧跟踪注释，专门用于支持以车辆为中心的追踪方法的评估。 通过广泛的实验结果证明，所提出的这种方法在新引入的数据集以及几个公共基准测试上都取得了稳健的表现，突显了其在通用物体追踪方面的有效性。尽管我们的框架设计用于多类别的泛化跟踪，但在传统的基准测试中也表现出色，在MOT17和MOT20测试集中分别获得了66.4和65.7的HOTA评分。 代码及数据集可在以下链接获取：[具体URL请根据实际情况插入]。
URL
https://arxiv.org/abs/2508.14370
PDF
-
A CLIP-based Uncertainty Modal Modeling Framework for Pedestrian Re-Identification in Autonomous Driving
2025-08-15 04:50:27Jialin Li, Shuqi Wu, Ning WangAbstract
Re-Identification (ReID) is a critical technology in intelligent perception systems, especially within autonomous driving, where onboard cameras must identify pedestrians across views and time in real-time to support safe navigation and trajectory prediction. However, the presence of uncertain or missing input modalities--such as RGB, infrared, sketches, or textual descriptions--poses significant challenges to conventional ReID approaches. While large-scale pre-trained models offer strong multimodal semantic modeling capabilities, their computational overhead limits practical deployment in resource-constrained environments. To address these challenges, we propose a lightweight Uncertainty Modal Modeling (UMM) framework, which integrates a multimodal token mapper, synthetic modality augmentation strategy, and cross-modal cue interactive learner. Together, these components enable unified feature representation, mitigate the impact of missing modalities, and extract complementary information across different data types. Additionally, UMM leverages CLIP's vision-language alignment ability to fuse multimodal inputs efficiently without extensive finetuning. Experimental results demonstrate that UMM achieves strong robustness, generalization, and computational efficiency under uncertain modality conditions, offering a scalable and practical solution for pedestrian re-identification in autonomous driving scenarios.
Abstract (translated)
重新识别（ReID）技术在智能感知系统中至关重要，尤其是在自主驾驶领域。车载摄像头需要实时识别不同视角和时间点上的行人，以支持安全导航和轨迹预测。然而，不确定或缺失的输入模态——例如RGB图像、红外图像、素描图或文本描述——对传统ReID方法构成了重大挑战。虽然大规模预训练模型提供了强大的多模态语义建模能力，但其计算开销限制了在资源受限环境中的实际部署。 为了解决这些问题，我们提出了一种轻量级的不确定性模式建模（UMM）框架，该框架整合了一个多模态令牌映射器、合成模态增强策略和跨模态线索互动学习者。这些组件共同作用，使得在不同数据类型间能够统一特征表示、减轻缺失模态的影响并提取互补信息成为可能。此外，UMM利用CLIP的视觉-语言对齐能力来高效融合多模态输入，无需进行大量微调。 实验结果表明，在不确定模态条件下，UMM框架具备强大的鲁棒性、泛化能力和计算效率，为自主驾驶场景中的行人重新识别提供了一个可扩展且实用的解决方案。
URL
https://arxiv.org/abs/2508.11218
PDF
-
TrackOR: Towards Personalized Intelligent Operating Rooms Through Robust Tracking
2025-08-11 13:28:50Tony Danjun Wang, Christian Heiliger, Nassir Navab, Lennart BastianAbstract
Providing intelligent support to surgical teams is a key frontier in automated surgical scene understanding, with the long-term goal of improving patient outcomes. Developing personalized intelligence for all staff members requires maintaining a consistent state of who is located where for long surgical procedures, which still poses numerous computational challenges. We propose TrackOR, a framework for tackling long-term multi-person tracking and re-identification in the operating room. TrackOR uses 3D geometric signatures to achieve state-of-the-art online tracking performance (+11% Association Accuracy over the strongest baseline), while also enabling an effective offline recovery process to create analysis-ready trajectories. Our work shows that by leveraging 3D geometric information, persistent identity tracking becomes attainable, enabling a critical shift towards the more granular, staff-centric analyses required for personalized intelligent systems in the operating room. This new capability opens up various applications, including our proposed temporal pathway imprints that translate raw tracking data into actionable insights for improving team efficiency and safety and ultimately providing personalized support.
Abstract (translated)
为外科团队提供智能支持是自动化手术场景理解的关键前沿领域，长期目标是改善患者结果。为了所有工作人员开发个性化智能需要保持长时间手术过程中每个人员的位置状态，这仍然面临众多计算挑战。我们提出了TrackOR框架，用于解决手术室内的长期多人跟踪和重识别问题。TrackOR利用3D几何签名实现了最先进的在线跟踪性能（比最强基线提高了+11%的关联准确性），同时还能实现有效的离线恢复过程以创建可用于分析的轨迹。 我们的工作表明，通过利用三维几何信息，持续身份跟踪变得可行，这使得向更细致、以工作人员为中心的分析转变成为可能，这对于手术室内的个性化智能系统至关重要。这种新能力开启了许多应用，包括我们提出的时序路径印记，该技术可以将原始跟踪数据转化为提高团队效率和安全性的可操作见解，并最终提供个性化的支持。
URL
https://arxiv.org/abs/2508.07968
PDF
