Abstract
The task of partial scene text retrieval involves localizing and searching for text instances that are the same or similar to a given query text from an image gallery. However, existing methods can only handle text-line instances, leaving the problem of searching for partial patches within these text-line instances unsolved due to a lack of patch annotations in the training data. To address this issue, we propose a network that can simultaneously retrieve both text-line instances and their partial patches. Our method embeds the two types of data (query text and scene text instances) into a shared feature space and measures their cross-modal similarities. To handle partial patches, our proposed approach adopts a Multiple Instance Learning (MIL) approach to learn their similarities with query text, without requiring extra annotations. However, constructing bags, which is a standard step of conventional MIL approaches, can introduce numerous noisy samples for training, and lower inference speed. To address this issue, we propose a Ranking MIL (RankMIL) approach to adaptively filter those noisy samples. Additionally, we present a Dynamic Partial Match Algorithm (DPMA) that can directly search for the target partial patch from a text-line instance during the inference stage, without requiring bags. This greatly improves the search efficiency and the performance of retrieving partial patches. The source code and dataset are available at this https URL.
Abstract (translated)
部分场景文本检索的任务涉及从图像库中定位和搜索与给定查询文本相同或相似的文本实例。然而，现有的方法只能处理整行文本实例，由于训练数据中缺乏补丁标注，导致在这些文本行实例内搜索部分补丁的问题仍未解决。为了解决这一问题，我们提出了一种能够同时检索文本行实例及其部分补丁的网络。我们的方法将两类数据（查询文本和场景文本实例）嵌入到一个共享特征空间，并测量它们之间的跨模态相似性。为了处理部分补丁，我们采用多示例学习（MIL）的方法来学习其与查询文本的相似性，而无需额外标注。然而，构建包袋——这是传统MIL方法的标准步骤——可能会引入大量噪音样本进行训练，并降低推理速度。为了解决这一问题，我们提出了一种排名多实例学习（RankMIL）方法来自适应地过滤这些噪音样本。此外，我们还提出了动态部分匹配算法（DPMA），该算法可以在推理阶段直接从文本行实例中搜索目标部分补丁，而无需构建包袋。这大大提高了搜索效率和检索部分补丁的性能。源代码和数据集可在[此处](https://www.example.com)获得（请将此链接替换为实际提供的URL）。
URL
https://arxiv.org/abs/2411.10261
