Abstract
Scene text retrieval has made significant progress with the assistance of accurate text localization. However, existing approaches typically require costly bounding box annotations for training. Besides, they mostly adopt a customized retrieval strategy but struggle to unify various types of queries to meet diverse retrieval needs. To address these issues, we introduce Muti-query Scene Text retrieval with Attention Recycling (MSTAR), a box-free approach for scene text retrieval. It incorporates progressive vision embedding to dynamically capture the multi-grained representation of texts and harmonizes free-style text queries with style-aware instructions. Additionally, a multi-instance matching module is integrated to enhance vision-language alignment. Furthermore, we build the Multi-Query Text Retrieval (MQTR) dataset, the first benchmark designed to evaluate the multi-query scene text retrieval capability of models, comprising four query types and 16k images. Extensive experiments demonstrate the superiority of our method across seven public datasets and the MQTR dataset. Notably, MSTAR marginally surpasses the previous state-of-the-art model by 6.4% in MAP on Total-Text while eliminating box annotation costs. Moreover, on the MQTR benchmark, MSTAR significantly outperforms the previous models by an average of 8.5%. The code and datasets are available at this https URL.
Abstract (translated)
场景文本检索在精确的文字定位技术的帮助下取得了显著进步。然而，现有的方法通常需要昂贵的边界框标注来进行训练。此外，这些方法大多采用定制化的检索策略，但在统一各种类型的查询以满足多样化的检索需求方面存在困难。为了解决这些问题，我们引入了带有注意力循环（Attention Recycling）的多查询场景文本检索(MSTAR)，这是一种无需边界的场景文本检索方法。MSTAR采用了渐进式视觉嵌入技术，能够动态捕捉文本的多层次表示，并将自由风格的文字查询与风格感知指令和谐统一。此外，还集成了一个多实例匹配模块以增强视觉语言对齐。 为了评估模型在多查询场景文本检索能力上的表现，我们构建了多查询文本检索(MQTR)数据集，这是首个专门为此任务设计的基准测试平台，包含四种类型的查询和16,000张图像。广泛的实验表明，在七个公开的数据集和MQTR数据集上，我们的方法均表现出优越性。值得注意的是，MSTAR在Total-Text数据集中以MAP（平均精度）为指标超越了前最先进模型6.4%，同时消除了边界框标注的成本。此外，在MQTR基准测试中，MSTAR的性能显著优于之前的模型，平均高出8.5%。 代码和数据集可以在以下链接获取：[此URL]（请将此处的"this https URL"替换为实际链接）。
URL
https://arxiv.org/abs/2506.10609
