Abstract
Pixel-level Scene Understanding is one of the fundamental problems in computer vision, which aims at recognizing object classes, masks and semantics of each pixel in the given image. Compared with image scene parsing, video scene parsing introduces temporal information, which can effectively improve the consistency and accuracy of prediction,because the real-world is actually video-based rather than a static state. In this paper, we adopt semi-supervised video semantic segmentation method based on unreliable pseudo labels. Then, We ensemble the teacher network model with the student network model to generate pseudo labels and retrain the student network. Our method achieves the mIoU scores of 63.71% and 67.83% on development test and final test respectively. Finally, we obtain the 1st place in the Video Scene Parsing in the Wild Challenge at CVPR 2024.
Abstract (translated)
像素级别的场景理解是计算机视觉中的一个基本问题，旨在识别给定图像中每个像素的对象类、掩码和语义。与图像场景解析相比，视频场景解析引入了时间信息，可以有效提高预测的一致性和准确性，因为现实世界实际上是基于视频的，而不是静态的状态。在本文中，我们采用了基于不可靠伪标签的半监督视频语义分割方法。然后，我们将教师网络模型与学生网络模型集成，生成伪标签并重新训练学生网络。我们的方法在开发测试和最终测试中的mIoU得分分别为63.71%和67.83%。最后，我们在CVPR 2024上的视频场景解析挑战中获得了第一名的成绩。
URL
https://arxiv.org/abs/2406.00587
