Abstract
Multi-object multi-part scene segmentation is a challenging task whose complexity scales exponentially with part granularity and number of scene objects. To address the task, we propose a plug-and-play approach termed OLAF. First, we augment the input (RGB) with channels containing object-based structural cues (fg/bg mask, boundary edge mask). We propose a weight adaptation technique which enables regular (RGB) pre-trained models to process the augmented (5-channel) input in a stable manner during optimization. In addition, we introduce an encoder module termed LDF to provide low-level dense feature guidance. This assists segmentation, particularly for smaller parts. OLAF enables significant mIoU gains of $\mathbf{3.3}$ (Pascal-Parts-58), $\mathbf{3.5}$ (Pascal-Parts-108) over the SOTA model. On the most challenging variant (Pascal-Parts-201), the gain is $\mathbf{4.0}$. Experimentally, we show that OLAF's broad applicability enables gains across multiple architectures (CNN, U-Net, Transformer) and datasets. The code is available at this http URL
Abstract (translated)
多对象多部件场景分割是一项具有挑战性的任务，其复杂性随零件的细致程度和场景中对象数量呈指数级增长。为了解决这一问题，我们提出了一种可插拔的方法，称为OLAF。首先，我们在输入（RGB）中加入包含基于物体结构线索的通道（前景/背景掩模、边界边缘掩模）。我们提出了一种权重适应技术，使常规预训练模型在优化过程中能够稳定处理增强后的5通道输入。此外，我们引入了一个编码模块LDF，以提供低级密集特征指导，这有助于分割，特别是对较小的部分来说更为有效。OLAF使得mIoU得分显著提升，在Pascal-Parts-58数据集上提升了$\mathbf{3.3}$，在Pascal-Parts-108数据集上提升了$\mathbf{3.5}$，超过现有最佳模型的表现。在最具挑战性的变体（Pascal-Parts-201）上，提升更是达到了$\mathbf{4.0}$。实验表明，OLAF的广泛适用性使其能够在多种架构（CNN、U-Net、Transformer）和数据集上取得提升。代码可以在以下网址获取：[此HTTP URL]
URL
https://arxiv.org/abs/2411.02858
