Abstract
Given the popularity of 360° images on social media platforms, 360° image compression becomes a critical technology for media storage and transmission. Conventional 360° image compression pipeline projects the spherical image into a single 2D plane, leading to issues of oversampling and distortion. In this paper, we propose a novel viewport-based neural compression pipeline for 360° images. By replacing the image projection in conventional 360° image compression pipelines with viewport extraction and efficiently compressing multiple viewports, the proposed pipeline minimizes the inherent oversampling and distortion issues. However, viewport extraction impedes information sharing between multiple viewports during compression, causing the loss of global information about the spherical image. To tackle this global information loss, we design a neural viewport codec to capture global prior information across multiple viewports and maximally compress the viewport data. The viewport codec is empowered by a transformer-based ViewPort ConText (VPCT) module that can be integrated with canonical learning-based 2D image compression structures. We compare the proposed pipeline with existing 360° image compression models and conventional 360° image compression pipelines building on learning-based 2D image codecs and standard hand-crafted codecs. Results show that our pipeline saves an average of $14.01\%$ bit consumption compared to the best-performing 360° image compression methods without compromising quality. The proposed VPCT-based codec also outperforms existing 2D image codecs in the viewport-based neural compression pipeline. Our code can be found at: this https URL.
Abstract (translated)
鉴于360°图像在社交媒体平台上的流行，360°图像压缩成为媒体存储与传输的关键技术。传统的360°图像压缩流程将球形图像投影至单一二维平面，导致过采样与失真问题。本文提出一种新颖的基于视口的神经压缩流程，通过用视口提取替代传统流程中的图像投影，并对多个视口进行高效压缩，从而最小化固有的过采样与失真问题。然而，视口提取阻碍了压缩过程中多个视口间的信息共享，导致球形图像的全局信息丢失。为应对此全局信息损失，我们设计了神经视口编解码器以捕获跨多个视口的全局先验信息，并最大程度压缩视口数据。该视口编解码器依托于基于Transformer的视口上下文（VPCT）模块，该模块可集成至基于学习的标准2D图像压缩结构中。我们将所提流程与现有360°图像压缩模型及基于学习的2D图像编解码器与标准手工设计编解码器构建的传统360°图像压缩流程进行对比。结果表明，在不损失质量的前提下，我们的流程平均节省了14.01%的比特消耗。此外，所提基于VPCT的编解码器在基于视口的神经压缩流程中也优于现有2D图像编解码器。代码链接为：此https URL。
URL
https://arxiv.org/abs/2603.22776
