Abstract
Accurate surgical instrument segmentation in endoscopic videos is crucial for computer-assisted interventions, yet remains challenging due to frequent occlusions, rapid motion, specular artefacts, and long-term instrument re-entry. While SAM3 provides a powerful spatio-temporal framework for video object segmentation, its performance in surgical scenes is limited by indiscriminate memory updates, fixed memory capacity, and weak identity recovery after occlusions. We propose ReMeDI-SAM3, a training-free memory-enhanced extension of SAM3, that addresses these limitations through three components: (i) relevance-aware memory filtering with a dedicated occlusion-aware memory for storing pre-occlusion frames, (ii) a piecewise interpolation scheme that expands the effective memory capacity, and (iii) a feature-based re-identification module with temporal voting for reliable post-occlusion identity disambiguation. Together, these components mitigate error accumulation and enable reliable recovery after occlusions. Evaluations on EndoVis17 and EndoVis18 under a zero-shot setting show absolute mcIoU improvements of around 7% and 16%, respectively, over vanilla SAM3, outperforming even prior training-based approaches. Project page: this https URL.
Abstract (translated)
在内窥镜视频中精确分割手术器械对于计算机辅助干预至关重要，但由于频繁的遮挡、快速移动、镜面伪影和长时间工具重新进入等问题，这一任务仍然具有挑战性。虽然SAM3提供了一种强大的时空框架来进行视频对象分割，但在外科场景中的性能受限于不分青红皂白的记忆更新、固定内存容量以及遮挡后身份恢复能力较弱的问题。 为此，我们提出了ReMeDI-SAM3，这是一个无需训练的增强记忆版本的SAM3扩展，通过三个组成部分来解决上述限制：（i）相关性感知的记忆过滤机制和一个专门用于存储被遮挡前帧的遮挡意识内存；（ii）一种分段插值方案以扩大有效内存容量；以及（iii）一种基于特征的身份重新识别模块，并结合时间投票技术，以便在遮挡后可靠地区分身份。这些组件共同作用减少错误积累，并支持在遮挡后的可靠恢复。 实验结果表明，在EndoVis17和EndoVis18数据集上的零样本设置下，ReMeDI-SAM3相较于基础版本的SAM3分别取得了大约7%和16%的绝对mcIoU改进，甚至超过了基于训练的方法的表现。项目主页：[请参见原文链接]。 这一研究成果展示了在不进行额外训练的前提下通过增强记忆机制提高视频分割性能的可能性，并为未来的研究提供了一个有价值的框架。
URL
https://arxiv.org/abs/2512.16880
