Abstract
The rapid progress of large Vision-Language Models (VLMs) has enabled a wide range of applications, such as image understanding and Visual Question Answering (VQA). Query images are often uploaded to the cloud, where VLMs are typically hosted, hence efficient image compression becomes crucial. However, traditional human-centric codecs are suboptimal in this setting because they preserve many task-irrelevant details. Existing Image Coding for Machines (ICM) methods also fall short, as they assume a fixed set of downstream tasks and cannot adapt to prompt-driven VLMs with an open-ended variety of objectives. We propose a lightweight, plug-and-play, prompt-guided prefiltering module to identify image regions most relevant to the text prompt, and consequently to the downstream task. The module preserves important details while smoothing out less relevant areas to improve compression efficiency. It is codec-agnostic and can be applied before conventional and learned encoders. Experiments on several VQA benchmarks show that our approach achieves a 25-50% average bitrate reduction while maintaining the same task accuracy. Our source code is available at this https URL.
Abstract (translated)
大型视觉语言模型（VLMs）的快速发展已催生了图像理解与视觉问答（VQA）等广泛应用。查询图像通常需上传至云端——VLMs 通常部署于此——因此高效的图像压缩至关重要。然而，传统面向人眼的编解码器在此场景下并非最优，因其保留了大量与任务无关的细节。现有面向机器的图像编码（ICM）方法也存在局限：它们预设固定的下游任务集，无法适应目标开放的提示驱动型 VLMs。我们提出一个轻量级、即插即用、由提示引导的预过滤模块，用于识别与文本提示（进而与下游任务）最相关的图像区域。该模块在保留关键细节的同时平滑处理相关性较低的区域，以提升压缩效率。其设计独立于具体编解码器，可应用于传统编码器与学习型编码器前。在多个 VQA 基准测试上的实验表明，该方法在保持相同任务准确率的前提下，实现了平均 25%-50% 的码率降低。源代码已公开于此链接。
URL
https://arxiv.org/abs/2604.00314
