Abstract
Large-scale Vision-Language Models (VLMs) such as CLIP learn powerful semantic representations but operate in Euclidean space, which fails to capture the inherent hierarchical structure of visual and linguistic concepts. Hyperbolic geometry, with its exponential volume growth, offers a principled alternative for embedding such hierarchies with low distortion. However, existing hyperbolic VLMs use entailment losses that are unstable: as parent embeddings contract toward the origin, their entailment cones widen toward a half-space, causing catastrophic cone collapse that destroys the intended hierarchy. Additionally, hierarchical evaluation of these models remains unreliable, being largely retrieval-based and correlation-based metrics and prone to taxonomy dependence and ambiguous negatives. To address these limitations, we propose an adaptive entailment loss paired with a norm regularizer that prevents cone collapse without heuristic aperture clipping. We further introduce an angle-based probabilistic entailment protocol (PEP) for evaluating hierarchical understanding, scored with AUC-ROC and Average Precision. This paper introduces a stronger hyperbolic VLM baseline ARGENT, Adaptive hieRarchical imaGe-tExt represeNTation. ARGENT improves the SOTA hyperbolic VLM by 0.7, 1.1, and 0.8 absolute points on image classification, text-to-image retrieval, and proposed hierarchical metrics, respectively.
Abstract (translated)
大规模视觉语言模型（如CLIP）能够学习强大的语义表征，但其在欧几里得空间中运作，无法捕捉视觉与语言概念固有的层次结构。双曲几何凭借其指数级增长的体积，为以低失真嵌入此类层次提供了原理性的替代方案。然而，现有双曲视觉语言模型所使用的蕴含损失存在不稳定性：当父级嵌入向原点收缩时，其蕴含锥会半空间方向扩张，导致灾难性的锥体塌陷，从而破坏预设的层次结构。此外，这些模型的层次化评估仍不可靠，主要依赖基于检索和相关性指标的方法，易受分类体系依赖与模糊负样本的影响。 为应对上述局限，本文提出一种自适应蕴含损失配合范数正则化器，可在无需启发式孔径裁剪的情况下防止锥体塌陷。同时引入基于角度的概率蕴含评估协议（PEP），采用AUC-ROC与平均精度进行评分。本研究提出了更优的双曲视觉语言模型基线ARGENT（自适应分层图像-文本表征）。ARGENT在图像分类、文本到图像检索以及提出的层次化指标上，分别较当前最优双曲视觉语言模型提升0.7、1.1和0.8个绝对点数。
URL
https://arxiv.org/abs/2603.23311
