Abstract
One of the key issues in Deep Neural Networks (DNNs) is the black-box nature of their internal feature extraction process. Targeting vision-related domains, this paper focuses on analysing the feature space of a DNN by proposing a decoder that can generate images whose features are guaranteed to closely match a user-specified feature. Owing to this guarantee that is missed in past studies, our decoder allows us to evidence which of various attributes in an image are encoded into a feature by the DNN, by generating images whose features are in proximity to that feature. Our decoder is implemented as a guided diffusion model that guides the reverse image generation of a pre-trained diffusion model to minimise the Euclidean distance between the feature of a clean image estimated at each step and the user-specified feature. One practical advantage of our decoder is that it can analyse feature spaces of different DNNs with no additional training and run on a single COTS GPU. The experimental results targeting CLIP's image encoder, ResNet-50 and vision transformer demonstrate that images generated by our decoder have features remarkably similar to the user-specified ones and reveal valuable insights into these DNNs' feature spaces.
Abstract (translated)
深度神经网络（DNN）的一个关键问题是其内部特征提取过程的黑盒性质。针对视觉相关领域，本文通过提出一种解码器来分析DNN的特征空间，该解码器可以生成具有与用户指定特性紧密匹配的图像特征的图像。由于以往的研究中忽略了这一保证，我们的解码器能够生成接近特定特性的图像，从而揭示各种属性在图像中的编码情况。我们的解码器实现为一种引导扩散模型，它指导预训练的扩散模型进行逆向图像生成过程，以最小化每一步估计的清洁图像特征与用户指定特征之间的欧氏距离。 我们解码器的一个实际优势是它可以无需额外训练地分析不同DNN的特征空间，并且可以在单个商用现成（COTS）GPU上运行。针对CLIP的图像编码器、ResNet-50和视觉变压器进行的实验结果表明，由我们的解码器生成的图像具有与用户指定特性非常相似的特征，并揭示了这些DNN特征空间中的宝贵见解。
URL
https://arxiv.org/abs/2509.07936
