Abstract
Many real-world applications in digital forensics, urban monitoring, and environmental analysis require jointly reasoning about visual appearance, geolocation, and time. Beyond standard geo-localization and time-of-capture prediction, these applications increasingly demand more complex capabilities, such as retrieving an image captured at the same location as a query image but at a specified target time. We formalize this problem as Geo-Time Aware Image Retrieval and curate a diverse benchmark of 4.5M paired image-location-time triplets for training and 86k high-quality triplets for evaluation. We then propose TIGeR, a multi-modal-transformer-based model that maps image, geolocation, and time into a unified geo-temporal embedding space. TIGeR supports flexible input configurations (single-modality and multi-modality queries) and uses the same representation to perform (i) geo-localization, (ii) time-of-capture prediction, and (iii) geo-time-aware retrieval. By better preserving underlying location identity under large appearance changes, TIGeR enables retrieval based on where and when a scene is, rather than purely on visual similarity. Extensive experiments show that TIGeR consistently outperforms strong baselines and state-of-the-art methods by up to 16% on time-of-year, 8% time-of-day prediction, and 14% in geo-time aware retrieval recall, highlighting the benefits of unified geo-temporal modeling.
Abstract (translated)
数字取证、城市监控和环境分析中的许多实际应用需要联合推理视觉外观、地理位置和时间信息。除标准地理定位和拍摄时间预测外，这些应用日益需要更复杂的能力，例如检索与查询图像拍摄于同一地点但在指定目标时间拍摄的图像。我们将此问题形式化为地理-时间感知图像检索，并构建了包含450万组图像-位置-时间三元组的多样化训练基准，以及8.6万组高质量三元组的评估基准。随后我们提出TIGeR，一种基于多模态Transformer的模型，可将图像、地理位置和时间映射到统一的地理-时间联合嵌入空间。TIGeR支持灵活的输入配置（单模态与多模态查询），并使用同一表征实现（i）地理定位、（ii）拍摄时间预测及（iii）地理-时间感知检索。通过在显著外观变化下更好地保持底层位置身份一致性，TIGeR实现了基于场景"何时何地"而非纯粹视觉相似度的检索。大量实验表明，TIGeR在季节时间预测上最高提升16%，昼夜时间预测提升8%，地理-时间感知检索召回率提升14%，持续优于强基线及现有先进方法，凸显了统一地理-时间建模的优势。
URL
https://arxiv.org/abs/2603.24749
