Abstract
Standard semantic instance segmentation provides useful, but inherently 2D information from a single image. To enable 3D analysis, one usually integrates absolute monocular depth estimation with instance segmentation. However, monocular depth is a difficult task. Instead, we leverage a simpler single-image task, occlusion-based relative depth ordering, providing coarser but useful 3D information. We show that relative depth ordering works more reliably from occlusions than from absolute depth. We propose to solve the joint task of relative depth ordering and segmentation of instances based on occlusions. We call this task Occlusion-Ordered Semantic Instance Segmentation (OOSIS). We develop an approach to OOSIS that extracts instances and their occlusion order simultaneously from oriented occlusion boundaries and semantic segmentation. Unlike popular detect-and-segment framework for instance segmentation, combining occlusion ordering with instance segmentation allows a simple and clean formulation of OOSIS as a labeling problem. As a part of our solution for OOSIS, we develop a novel oriented occlusion boundaries approach that significantly outperforms prior work. We also develop a new joint OOSIS metric based both on instance mask accuracy and correctness of their occlusion order. We achieve better performance than strong baselines on KINS and COCOA datasets.
Abstract (translated)
标准语义实例分割从单张图像中提供有用但本质上是二维的信息。为了实现三维分析，通常会将绝对单目深度估计与实例分割相结合。然而，单目深度估计是一个极具挑战性的任务。因此，我们采用了一个更简单的基于单一图像的任务：基于遮挡的相对深度排序，这虽然提供了较粗略但有用的三维信息。我们展示了相对于绝对深度而言，从遮挡中获取相对深度排序更为可靠的方法。为此，我们提出了解决相对深度排序和基于遮挡的实例分割联合任务的问题，并将这项任务命名为“遮挡顺序语义实例分割”（Occlusion-Ordered Semantic Instance Segmentation, OOSIS）。 为解决OOSIS问题，我们开发了一种方法，该方法能够从定向遮挡边界和语义分割中同时提取实例及其遮挡顺序。不同于流行的检测与分割框架用于实例分割，将遮挡排序与实例分割结合可以使OOSIS作为一个标记问题得到简单而清晰的表述。 作为解决OOSIS方案的一部分，我们开发了一种新颖的定向遮挡边界方法，该方法在性能上显著优于先前的工作。同时，我们还提出了一种新的联合OOSIS指标，它不仅考虑了实例掩码的准确性，也考量了它们遮挡顺序的正确性。 我们在KINS和COCOA数据集上实现了比强基线更好的性能。
URL
https://arxiv.org/abs/2504.14054
