Abstract
Cross-view geo-localization identifies the locations of street-view images by matching them with geo-tagged satellite images or OSM. However, most studies focus on image-to-image retrieval, with fewer addressing text-guided retrieval, a task vital for applications like pedestrian navigation and emergency response. In this work, we introduce a novel task for cross-view geo-localization with natural language descriptions, which aims to retrieve corresponding satellite images or OSM database based on scene text. To support this task, we construct the CVG-Text dataset by collecting cross-view data from multiple cities and employing a scene text generation approach that leverages the annotation capabilities of Large Multimodal Models to produce high-quality scene text descriptions with localization this http URL, we propose a novel text-based retrieval localization method, CrossText2Loc, which improves recall by 10% and demonstrates excellent long-text retrieval capabilities. In terms of explainability, it not only provides similarity scores but also offers retrieval reasons. More information can be found at this https URL.
Abstract (translated)
跨视角地理定位通过将街景图像与带地理标签的卫星图像或OSM（开放街道地图）进行匹配来识别其位置。然而，大多数研究集中于图像到图像的检索，较少关注文本引导的检索任务，而这一任务对于行人导航和紧急响应等应用至关重要。在本项工作中，我们引入了一种新的跨视角地理定位任务，该任务使用自然语言描述来检索相应的卫星图像或OSM数据库，基于场景中的文字内容进行匹配。为了支持这项任务，我们通过收集来自多个城市的跨视角数据并采用一种利用大型多模态模型的标注能力生成高质量场景文本描述的方法构建了CVG-Text数据集（详情请参见此链接：[http://this%20https%20URL]）。在此基础上，我们提出了一种新颖的文字检索定位方法——CrossText2Loc，该方法能将召回率提高10%，并且展示了出色的长文本检索能力。在解释性方面，它不仅提供了相似度得分，还给出了检索原因。更多详情请参见此链接：[https://this%20https%20URL]。 注释：原文中的两个“this https URL”指向具体的网址信息，在翻译中保留了这种表示方式，实际应用时应替换为具体的网络地址。
URL
https://arxiv.org/abs/2412.17007
