Scene_Text
-
What Shape Is Optimal for Masks in Text Removal?
2025-11-27 14:34:35Hyakka Nakada, Marika KubotaAbstract
The advent of generative models has dramatically improved the accuracy of image inpainting. In particular, by removing specific text from document images, reconstructing original images is extremely important for industrial applications. However, most existing methods of text removal focus on deleting simple scene text which appears in images captured by a camera in an outdoor environment. There is little research dedicated to complex and practical images with dense text. Therefore, we created benchmark data for text removal from images including a large amount of text. From the data, we found that text-removal performance becomes vulnerable against mask profile perturbation. Thus, for practical text-removal tasks, precise tuning of the mask shape is essential. This study developed a method to model highly flexible mask profiles and learn their parameters using Bayesian optimization. The resulting profiles were found to be character-wise masks. It was also found that the minimum cover of a text region is not optimal. Our research is expected to pave the way for a user-friendly guideline for manual masking.
Abstract (translated)
URL
https://arxiv.org/abs/2511.22499
PDF
-
SFA: Scan, Focus, and Amplify toward Guidance-aware Answering for Video TextVQA
2025-11-25 11:14:39Haibin He, Qihuang Zhong, Juhua Liu, Bo Du, Peng Wang, Jing ZhangAbstract
Video text-based visual question answering (Video TextVQA) task aims to answer questions about videos by leveraging the visual text appearing within the videos. This task poses significant challenges, requiring models to accurately perceive and comprehend scene text that varies in scale, orientation, and clarity across frames, while effectively integrating temporal and semantic context to generate precise answers. Moreover, the model must identify question-relevant textual cues and filter out redundant or irrelevant information to ensure answering is guided by the most relevant and informative cues. To address these challenges, we propose SFA, a training-free framework and the first Video-LLM-based method tailored for Video TextVQA, motivated by the human process of answering questions. By adaptively scanning video frames, selectively focusing on key regions, and directly amplifying them, SFA effectively guides the Video-LLM's attention toward essential cues, enabling it to generate more accurate answers. SFA achieves new state-of-the-art results across several public Video TextVQA datasets and surpasses previous methods by a substantial margin, demonstrating its effectiveness and generalizability.
Abstract (translated)
URL
https://arxiv.org/abs/2511.20190
PDF
-
Reading Between the Lines: Abstaining from VLM-Generated OCR Errors via Latent Representation Probes
2025-11-25 00:24:42Jihan Yao, Achin Kulshrestha, Nathalie Rauschmayr, Reed Roberts, Banghua Zhu, Yulia Tsvetkov, Federico TombariAbstract
As VLMs are deployed in safety-critical applications, their ability to abstain from answering when uncertain becomes crucial for reliability, especially in Scene Text Visual Question Answering (STVQA) tasks. For example, OCR errors like misreading "50 mph" as "60 mph" could cause severe traffic accidents. This leads us to ask: Can VLMs know when they can't see? Existing abstention methods suggest pessimistic answers: they either rely on miscalibrated output probabilities or require semantic agreement unsuitable for OCR tasks. However, this failure may indicate we are looking in the wrong place: uncertainty signals could be hidden in VLMs' internal representations. Building on this insight, we propose Latent Representation Probing (LRP): training lightweight probes on hidden states or attention patterns. We explore three probe designs: concatenating representations across all layers, aggregating attention over visual tokens, and ensembling single layer probes by majority vote. Experiments on four benchmarks across image and video modalities show LRP improves abstention accuracy by 7.6\% over best baselines. Our analysis reveals: probes generalize across various uncertainty sources and datasets, and optimal signals emerge from intermediate rather than final layers. This establishes a principled framework for building deployment-ready AI systems by detecting confidence signals from internal states rather than unreliable outputs.
Abstract (translated)
URL
https://arxiv.org/abs/2511.19806
PDF
-
One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolution
2025-11-21 11:00:59Yushun Fang, Yuxiang Chen, Shibo Yin, Qiang Hu, Jiangchao Yao, Ya Zhang, Xiaoyun Zhang, Yanfeng WangAbstract
Recent advances in diffusion-based real-world image super-resolution (Real-ISR) have demonstrated remarkable perceptual quality, yet the balance between fidelity and controllability remains a problem: multi-step diffusion-based methods suffer from generative diversity and randomness, resulting in low fidelity, while one-step methods lose control flexibility due to fidelity-specific finetuning. In this paper, we present ODTSR, a one-step diffusion transformer based on Qwen-Image that performs Real-ISR considering fidelity and controllability simultaneously: a newly introduced visual stream receives low-quality images (LQ) with adjustable noise (Control Noise), and the original visual stream receives LQs with consistent noise (Prior Noise), forming the Noise-hybrid Visual Stream (NVS) design. ODTSR further employs Fidelity-aware Adversarial Training (FAA) to enhance controllability and achieve one-step inference. Extensive experiments demonstrate that ODTSR not only achieves state-of-the-art (SOTA) performance on generic Real-ISR, but also enables prompt controllability on challenging scenarios such as real-world scene text image super-resolution (STISR) of Chinese characters without training on specific datasets.
Abstract (translated)
URL
https://arxiv.org/abs/2511.17138
PDF
-
ARC-Chapter: Structuring Hour-Long Videos into Navigable Chapters and Hierarchical Summaries
2025-11-18 10:53:14Junfu Pu, Teng Wang, Yixiao Ge, Yuying Ge, Chen Li, Ying ShanAbstract
The proliferation of hour-long videos (e.g., lectures, podcasts, documentaries) has intensified demand for efficient content structuring. However, existing approaches are constrained by small-scale training with annotations that are typical short and coarse, restricting generalization to nuanced transitions in long videos. We introduce ARC-Chapter, the first large-scale video chaptering model trained on over million-level long video chapters, featuring bilingual, temporally grounded, and hierarchical chapter annotations. To achieve this goal, we curated a bilingual English-Chinese chapter dataset via a structured pipeline that unifies ASR transcripts, scene texts, visual captions into multi-level annotations, from short title to long summaries. We demonstrate clear performance improvements with data scaling, both in data volume and label intensity. Moreover, we design a new evaluation metric termed GRACE, which incorporates many-to-one segment overlaps and semantic similarity, better reflecting real-world chaptering flexibility. Extensive experiments demonstrate that ARC-Chapter establishes a new state-of-the-art by a significant margin, outperforming the previous best by 14.0% in F1 score and 11.3% in SODA score. Moreover, ARC-Chapter shows excellent transferability, improving the state-of-the-art on downstream tasks like dense video captioning on YouCook2.
Abstract (translated)
URL
https://arxiv.org/abs/2511.14349
PDF
-
TripleFDS: Triple Feature Disentanglement and Synthesis for Scene Text Editing
2025-11-17 14:15:03Yuchen Bao, Yiting Wang, Wenjian Huang, Haowei Wang, Shen Chen, Taiping Yao, Shouhong Ding, Jianguo ZhangAbstract
Scene Text Editing (STE) aims to naturally modify text in images while preserving visual consistency, the decisive factors of which can be divided into three parts, i.e., text style, text content, and background. Previous methods have struggled with incomplete disentanglement of editable attributes, typically addressing only one aspect - such as editing text content - thus limiting controllability and visual consistency. To overcome these limitations, we propose TripleFDS, a novel framework for STE with disentangled modular attributes, and an accompanying dataset called SCB Synthesis. SCB Synthesis provides robust training data for triple feature disentanglement by utilizing the "SCB Group", a novel construct that combines three attributes per image to generate diverse, disentangled training groups. Leveraging this construct as a basic training unit, TripleFDS first disentangles triple features, ensuring semantic accuracy through inter-group contrastive regularization and reducing redundancy through intra-sample multi-feature orthogonality. In the synthesis phase, TripleFDS performs feature remapping to prevent "shortcut" phenomena during reconstruction and mitigate potential feature leakage. Trained on 125,000 SCB Groups, TripleFDS achieves state-of-the-art image fidelity (SSIM of 44.54) and text accuracy (ACC of 93.58%) on the mainstream STE benchmarks. Besides superior performance, the more flexible editing of TripleFDS supports new operations such as style replacement and background transfer. Code: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2511.13399
PDF
-
STELLAR: Scene Text Editor for Low-Resource Languages and Real-World Data
2025-11-13 05:18:03Yongdeuk Seo, Hyun-seok Min, Sungchul ChoiAbstract
Scene Text Editing (STE) is the task of modifying text content in an image while preserving its visual style, such as font, color, and background. While recent diffusion-based approaches have shown improvements in visual quality, key limitations remain: lack of support for low-resource languages, domain gap between synthetic and real data, and the absence of appropriate metrics for evaluating text style preservation. To address these challenges, we propose STELLAR (Scene Text Editor for Low-resource LAnguages and Real-world data). STELLAR enables reliable multilingual editing through a language-adaptive glyph encoder and a multi-stage training strategy that first pre-trains on synthetic data and then fine-tunes on real images. We also construct a new dataset, STIPLAR(Scene Text Image Pairs of Low-resource lAnguages and Real-world data), for training and evaluation. Furthermore, we propose Text Appearance Similarity (TAS), a novel metric that assesses style preservation by independently measuring font, color, and background similarity, enabling robust evaluation even without ground truth. Experimental results demonstrate that STELLAR outperforms state-of-the-art models in visual consistency and recognition accuracy, achieving an average TAS improvement of 2.2% across languages over the baselines.
Abstract (translated)
URL
https://arxiv.org/abs/2511.09977
PDF
-
OTSNet: A Neurocognitive-Inspired Observation-Thinking-Spelling Pipeline for Scene Text Recognition
2025-11-11 11:40:48Lixu Sun, Nurmemet Yolwas, Wushour SilamuAbstract
Scene Text Recognition (STR) remains challenging due to real-world complexities, where decoupled visual-linguistic optimization in existing frameworks amplifies error propagation through cross-modal misalignment. Visual encoders exhibit attention bias toward background distractors, while decoders suffer from spatial misalignment when parsing geometrically deformed text-collectively degrading recognition accuracy for irregular patterns. Inspired by the hierarchical cognitive processes in human visual perception, we propose OTSNet, a novel three-stage network embodying a neurocognitive-inspired Observation-Thinking-Spelling pipeline for unified STR modeling. The architecture comprises three core components: (1) a Dual Attention Macaron Encoder (DAME) that refines visual features through differential attention maps to suppress irrelevant regions and enhance discriminative focus; (2) a Position-Aware Module (PAM) and Semantic Quantizer (SQ) that jointly integrate spatial context with glyph-level semantic abstraction via adaptive sampling; and (3) a Multi-Modal Collaborative Verifier (MMCV) that enforces self-correction through cross-modal fusion of visual, semantic, and character-level features. Extensive experiments demonstrate that OTSNet achieves state-of-the-art performance, attaining 83.5% average accuracy on the challenging Union14M-L benchmark and 79.1% on the heavily occluded OST dataset-establishing new records across 9 out of 14 evaluation scenarios.
Abstract (translated)
URL
https://arxiv.org/abs/2511.08133
PDF
-
Hybrid CNN-ViT Framework for Motion-Blurred Scene Text Restoration
2025-11-08 17:48:58Umar Rashid (University of Engineering & Technology, New Campus, Lahore, Pakistan), Muhammad Arslan Arshad (University of Engineering & Technology, New Campus, Lahore, Pakistan), Ghulam Ahmad (University of Engineering & Technology, New Campus, Lahore, Pakistan), Muhammad Zeeshan Anjum (University of Engineering & Technology, New Campus, Lahore, Pakistan), Rizwan Khan (University of Engineering & Technology, New Campus, Lahore, Pakistan), Muhammad Akmal (Sheffield Hallam University, Sheffield, UK)Abstract
Motion blur in scene text images severely impairs readability and hinders the reliability of computer vision tasks, including autonomous driving, document digitization, and visual information retrieval. Conventional deblurring approaches are often inadequate in handling spatially varying blur and typically fall short in modeling the long-range dependencies necessary for restoring textual clarity. To overcome these limitations, we introduce a hybrid deep learning framework that combines convolutional neural networks (CNNs) with vision transformers (ViTs), thereby leveraging both local feature extraction and global contextual reasoning. The architecture employs a CNN-based encoder-decoder to preserve structural details, while a transformer module enhances global awareness through self-attention. Training is conducted on a curated dataset derived from TextOCR, where sharp scene-text samples are paired with synthetically blurred versions generated using realistic motion-blur kernels of multiple sizes and orientations. Model optimization is guided by a composite loss that incorporates mean absolute error (MAE), squared error (MSE), perceptual similarity, and structural similarity (SSIM). Quantitative eval- uations show that the proposed method attains 32.20 dB in PSNR and 0.934 in SSIM, while remaining lightweight with 2.83 million parameters and an average inference time of 61 ms. These results highlight the effectiveness and computational efficiency of the CNN-ViT hybrid design, establishing its practicality for real-world motion-blurred scene-text restoration.
Abstract (translated)
URL
https://arxiv.org/abs/2511.06087
PDF
-
Text-VQA Aug: Pipelined Harnessing of Large Multimodal Models for Automated Synthesis
2025-11-03 20:28:22Soham Joshi, Shwet Kamal Mishra, Viswanath GopalakrishnanAbstract
Creation of large-scale databases for Visual Question Answering tasks pertaining to the text data in a scene (text-VQA) involves skilful human annotation, which is tedious and challenging. With the advent of foundation models that handle vision and language modalities, and with the maturity of OCR systems, it is the need of the hour to establish an end-to-end pipeline that can synthesize Question-Answer (QA) pairs based on scene-text from a given image. We propose a pipeline for automated synthesis for text-VQA dataset that can produce faithful QA pairs, and which scales up with the availability of scene text data. Our proposed method harnesses the capabilities of multiple models and algorithms involving OCR detection and recognition (text spotting), region of interest (ROI) detection, caption generation, and question generation. These components are streamlined into a cohesive pipeline to automate the synthesis and validation of QA pairs. To the best of our knowledge, this is the first pipeline proposed to automatically synthesize and validate a large-scale text-VQA dataset comprising around 72K QA pairs based on around 44K images.
Abstract (translated)
URL
https://arxiv.org/abs/2511.02046
PDF
-
Restore Text First, Enhance Image Later: Two-Stage Scene Text Image Super-Resolution with Glyph Structure Guidance
2025-10-24 15:59:04Minxing Luo, Linlong Fan, Wang Qiushi, Ge Wu, Yiyan Luo, Yuhang Yu, Jinwei Chen, Yaxing Wang, Qingnan Fan, Jian YangAbstract
Current generative super-resolution methods show strong performance on natural images but distort text, creating a fundamental trade-off between image quality and textual readability. To address this, we introduce \textbf{TIGER} (\textbf{T}ext-\textbf{I}mage \textbf{G}uided sup\textbf{E}r-\textbf{R}esolution), a novel two-stage framework that breaks this trade-off through a \textit{"text-first, image-later"} paradigm. \textbf{TIGER} explicitly decouples glyph restoration from image enhancement: it first reconstructs precise text structures and then uses them to guide subsequent full-image super-resolution. This glyph-to-image guidance ensures both high fidelity and visual consistency. To support comprehensive training and evaluation, we also contribute the \textbf{UltraZoom-ST} (UltraZoom-Scene Text), the first scene text dataset with extreme zoom (\textbf{$\times$14.29}). Extensive experiments show that \textbf{TIGER} achieves \textbf{state-of-the-art} performance, enhancing readability while preserving overall image quality.
Abstract (translated)
URL
https://arxiv.org/abs/2510.21590
PDF
-
SceneTextStylizer: A Training-Free Scene Text Style Transfer Framework with Diffusion Model
2025-10-13 02:11:57Honghui Yuan, Keiji YanaiAbstract
With the rapid development of diffusion models, style transfer has made remarkable progress. However, flexible and localized style editing for scene text remains an unsolved challenge. Although existing scene text editing methods have achieved text region editing, they are typically limited to content replacement and simple styles, which lack the ability of free-style transfer. In this paper, we introduce SceneTextStylizer, a novel training-free diffusion-based framework for flexible and high-fidelity style transfer of text in scene images. Unlike prior approaches that either perform global style transfer or focus solely on textual content modification, our method enables prompt-guided style transformation specifically for text regions, while preserving both text readability and stylistic consistency. To achieve this, we design a feature injection module that leverages diffusion model inversion and self-attention to transfer style features effectively. Additionally, a region control mechanism is introduced by applying a distance-based changing mask at each denoising step, enabling precise spatial control. To further enhance visual quality, we incorporate a style enhancement module based on the Fourier transform to reinforce stylistic richness. Extensive experiments demonstrate that our method achieves superior performance in scene text style transformation, outperforming existing state-of-the-art methods in both visual fidelity and text preservation.
Abstract (translated)
随着扩散模型的快速发展，风格迁移取得了显著进展。然而，场景文本中的灵活且局部化的风格编辑仍然是一个未解决的挑战。尽管现有的场景文本编辑方法已经实现了文字区域编辑，但它们通常局限于内容替换和简单的样式更改，并缺乏自由式风格转移的能力。在本文中，我们介绍了SceneTextStylizer，这是一种新颖的、无需训练的基于扩散模型的框架，用于对场景图像中的文本进行灵活且高保真的风格转换。 与先前的方法要么执行全局风格迁移，要么仅专注于文本内容修改不同，我们的方法通过使用提示引导的方式实现了针对文字区域的特定风格变换，并同时保持了文本可读性和风格一致性。为了实现这一点，我们设计了一个特征注入模块，该模块利用扩散模型逆向工程和自注意力机制有效地转移风格特征。此外，我们引入了一种基于距离变化掩码的区域控制机制，在每次去噪步骤中应用以实现精确的空间控制。为了进一步提高视觉质量，我们还集成了一个基于傅立叶变换的风格增强模块，用于加强风格丰富性。 大量的实验表明，我们的方法在场景文本风格转换方面表现出了优越性能，在视觉保真度和文本保留方面均优于现有的最先进的方法。
URL
https://arxiv.org/abs/2510.10910
PDF
-
A Large-scale Dataset for Robust Complex Anime Scene Text Detection
2025-10-09 08:47:52Ziyi Dong, Yurui Zhang, Changmao Li, Naomi Rue Golding, Qing LongAbstract
Current text detection datasets primarily target natural or document scenes, where text typically appear in regular font and shapes, monotonous colors, and orderly layouts. The text usually arranged along straight or curved lines. However, these characteristics differ significantly from anime scenes, where text is often diverse in style, irregularly arranged, and easily confused with complex visual elements such as symbols and decorative patterns. Text in anime scene also includes a large number of handwritten and stylized fonts. Motivated by this gap, we introduce AnimeText, a large-scale dataset containing 735K images and 4.2M annotated text blocks. It features hierarchical annotations and hard negative samples tailored for anime scenarios. %Cross-dataset evaluations using state-of-the-art methods demonstrate that models trained on AnimeText achieve superior performance in anime text detection tasks compared to existing datasets. To evaluate the robustness of AnimeText in complex anime scenes, we conducted cross-dataset benchmarking using state-of-the-art text detection methods. Experimental results demonstrate that models trained on AnimeText outperform those trained on existing datasets in anime scene text detection tasks. AnimeText on HuggingFace: this https URL
Abstract (translated)
当前的文字检测数据集主要针对自然场景或文档场景，其中文本通常以规则字体和形状、单调的颜色以及有序的布局出现。这些场景中的文字一般沿直线或曲线排列。然而，在动漫场景中，上述特征有显著差异：文字风格多样、排布不规则，并且容易与复杂的视觉元素（如符号和装饰图案）混淆。此外，动漫场景中的文本包括大量手写字体和特殊字体。 为了填补这一空白，我们引入了AnimeText，这是一个大规模的数据集，包含735K张图片和4.2M个标注的文字块。该数据集具有层次化的注释，并针对动漫场景定制了硬负样本（hard negative samples）。使用最先进的方法进行跨数据集评估表明，在动漫文字检测任务中，基于AnimeText训练的模型优于现有数据集上的模型。 为了评估AnimeText在复杂动漫场景中的鲁棒性，我们使用最先进的文本检测方法进行了跨数据集基准测试。实验结果表明，在动漫场景的文字检测任务中，基于AnimeText训练的模型比基于其他现有数据集训练的模型表现出色。可以在HuggingFace上访问AnimeText：[此链接](https://huggingface.co/datasets/this-url)（请将“this-url”替换为实际的URL）。
URL
https://arxiv.org/abs/2510.07951
PDF
-
OTR: Synthesizing Overlay Text Dataset for Text Removal
2025-10-03 07:44:07Jan Zdenek, Wataru Shimoda, Kota YamaguchiAbstract
Text removal is a crucial task in computer vision with applications such as privacy preservation, image editing, and media reuse. While existing research has primarily focused on scene text removal in natural images, limitations in current datasets hinder out-of-domain generalization or accurate evaluation. In particular, widely used benchmarks such as SCUT-EnsText suffer from ground truth artifacts due to manual editing, overly simplistic text backgrounds, and evaluation metrics that do not capture the quality of generated results. To address these issues, we introduce an approach to synthesizing a text removal benchmark applicable to domains other than scene texts. Our dataset features text rendered on complex backgrounds using object-aware placement and vision-language model-generated content, ensuring clean ground truth and challenging text removal scenarios. The dataset is available at this https URL .
Abstract (translated)
文本移除是计算机视觉中的一个关键任务，具有隐私保护、图像编辑和媒体重用等多种应用。尽管现有研究主要集中在自然图像中场景文字的去除上，但当前数据集的局限性阻碍了跨域泛化或准确评估。尤其是广泛使用的基准如SCUT-EnsText，由于人工编辑导致的真实标签瑕疵、过于简单的文本背景以及无法捕捉生成结果质量的评价指标等问题。 为解决这些问题，我们提出了一种合成适用于除场景文字外其他领域的文本移除基准的方法。我们的数据集特点是使用对象感知放置和视觉语言模型生成的内容，在复杂背景下渲染文本，确保真实标签干净且提供挑战性的文本移除场景。该数据集可在此网址获取：[请在此处插入实际的URL链接]。
URL
https://arxiv.org/abs/2510.02787
PDF
-
LadderMoE: Ladder-Side Mixture of Experts Adapters for Bronze Inscription Recognition
2025-10-02 04:14:14Rixin Zhou, Peiqiang Qiu, Qian Zhang, Chuntao Li, Xi YangAbstract
Bronze inscriptions (BI), engraved on ritual vessels, constitute a crucial stage of early Chinese writing and provide indispensable evidence for archaeological and historical studies. However, automatic BI recognition remains difficult due to severe visual degradation, multi-domain variability across photographs, rubbings, and tracings, and an extremely long-tailed character distribution. To address these challenges, we curate a large-scale BI dataset comprising 22454 full-page images and 198598 annotated characters spanning 6658 unique categories, enabling robust cross-domain evaluation. Building on this resource, we develop a two-stage detection-recognition pipeline that first localizes inscriptions and then transcribes individual characters. To handle heterogeneous domains and rare classes, we equip the pipeline with LadderMoE, which augments a pretrained CLIP encoder with ladder-style MoE adapters, enabling dynamic expert specialization and stronger robustness. Comprehensive experiments on single-character and full-page recognition tasks demonstrate that our method substantially outperforms state-of-the-art scene text recognition baselines, achieving superior accuracy across head, mid, and tail categories as well as all acquisition modalities. These results establish a strong foundation for bronze inscription recognition and downstream archaeological analysis.
Abstract (translated)
青铜铭文（BI）刻在祭祀器皿上，构成了早期中国文字发展的一个重要阶段，并为考古学和历史研究提供了不可或缺的证据。然而，由于视觉退化严重、不同照片、拓片和描摹中的多领域变化以及极其长尾的文字分布，自动识别BI仍然具有挑战性。为了应对这些挑战，我们整理了一个大规模的数据集，包含22454张全页图像和198598个注释字符，涵盖了6658种独特的类别，从而能够进行稳健的跨领域评估。在此基础上，我们开发了一条两阶段检测-识别流水线，首先定位铭文，然后转录单个文字。为了处理异质领域和稀有类别的问题，我们在该流水线上配备了LadderMoE，这是一种通过添加类似梯度式的专家适配器来增强预训练的CLIP编码器的方法，从而实现动态专家专业化并提高更强的鲁棒性。在单一字符和全页识别任务上的全面实验表明，我们的方法显著优于现有的场景文本识别基线，在头部、中段和尾部类别以及所有获取模式下都取得了更高的准确性。这些结果为青铜铭文识别及下游考古分析奠定了坚实的基础。
URL
https://arxiv.org/abs/2510.01651
PDF
-
Human + AI for Accelerating Ad Localization Evaluation
2025-09-16 00:52:41Harshit Rajgarhia, Shivali Dalmia, Mengyang Zhao, Mukherji Abhishek, Kiran GaneshAbstract
Adapting advertisements for multilingual audiences requires more than simple text translation; it demands preservation of visual consistency, spatial alignment, and stylistic integrity across diverse languages and formats. We introduce a structured framework that combines automated components with human oversight to address the complexities of advertisement localization. To the best of our knowledge, this is the first work to integrate scene text detection, inpainting, machine translation (MT), and text reimposition specifically for accelerating ad localization evaluation workflows. Qualitative results across six locales demonstrate that our approach produces semantically accurate and visually coherent localized advertisements, suitable for deployment in real-world workflows.
Abstract (translated)
将广告适应为多语言受众的需求不仅仅局限于简单的文本翻译；这还需要在多种语言和格式下保持视觉一致性、空间对齐以及风格的完整性。我们提出了一种结构化的框架，该框架结合了自动化组件与人工监督，以解决广告本地化过程中遇到的各种复杂问题。据我们所知，这是首次将场景文本检测、图像修复（inpainting）、机器翻译（MT）和文本重新置入技术结合起来，专门用于加速广告本地化评估工作流程的研究。通过六个不同地区的定性结果证明，我们的方法能够生成语义准确且视觉上连贯的本地化广告，适合应用于实际的工作环境中。
URL
https://arxiv.org/abs/2509.12543
PDF
-
Detecting Text Manipulation in Images using Vision Language Models
2025-09-12 14:20:29Vidit Vidit, Pavel Korshunov, Amir Mohammadi, Christophe Ecabert, Ketan Kotwal, S\'ebastien MarcelAbstract
Recent works have shown the effectiveness of Large Vision Language Models (VLMs or LVLMs) in image manipulation detection. However, text manipulation detection is largely missing in these studies. We bridge this knowledge gap by analyzing closed- and open-source VLMs on different text manipulation datasets. Our results suggest that open-source models are getting closer, but still behind closed-source ones like GPT- 4o. Additionally, we benchmark image manipulation detection-specific VLMs for text manipulation detection and show that they suffer from the generalization problem. We benchmark VLMs for manipulations done on in-the-wild scene texts and on fantasy ID cards, where the latter mimic a challenging real-world misuse.
Abstract (translated)
最近的研究表明，大型视觉语言模型（VLM或LVLM）在图像篡改检测方面非常有效。然而，在这些研究中，对于文本篡改的检测则较少涉及。我们通过分析闭源和开源的VLM模型在不同文本篡改数据集上的表现来填补这一知识空白。我们的结果显示，尽管开源模型正在迎头赶上，但它们仍落后于如GPT-4o这样的闭源模型。此外，我们还针对专门用于图像篡改检测的VLM进行了基准测试，并展示了这些模型在文本篡改检测方面存在泛化问题。我们在野外场景文本和幻想身份证件上的操作中对VLM进行了基准测试，后者模拟了一个具有挑战性的现实世界滥用案例。
URL
https://arxiv.org/abs/2509.10278
PDF
-
TEACH: Text Encoding as Curriculum Hints for Scene Text Recognition
2025-08-02 02:28:09Xiahan Yang, Hui ZhengAbstract
Scene Text Recognition (STR) remains a challenging task due to complex visual appearances and limited semantic priors. We propose TEACH, a novel training paradigm that injects ground-truth text into the model as auxiliary input and progressively reduces its influence during training. By encoding target labels into the embedding space and applying loss-aware masking, TEACH simulates a curriculum learning process that guides the model from label-dependent learning to fully visual recognition. Unlike language model-based approaches, TEACH requires no external pretraining and introduces no inference overhead. It is model-agnostic and can be seamlessly integrated into existing encoder-decoder frameworks. Extensive experiments across multiple public benchmarks show that models trained with TEACH achieve consistently improved accuracy, especially under challenging conditions, validating its robustness and general applicability.
Abstract (translated)
场景文本识别（STR）由于复杂的视觉外观和有限的语义先验，仍是一项具有挑战性的任务。我们提出了一种新的训练范式TEACH，该方法在模型中注入真实的文本作为辅助输入，并在训练过程中逐步减少其影响。通过将目标标签编码到嵌入空间并通过损失感知掩码进行处理，TEACH模拟了渐进学习过程，引导模型从依赖于标签的学习转向完全基于视觉的识别。与基于语言模型的方法不同，TEACH不需要外部预训练并且不会增加推理开销。该方法是模型不可知的，并且可以无缝集成到现有的编码器-解码器框架中。 在多个公共基准测试中的广泛实验表明，使用TEACH进行训练的模型在各种条件下的一致性准确度得到了提高，尤其是在挑战性的环境中表现尤为出色，这验证了其鲁棒性和通用适用性。
URL
https://arxiv.org/abs/2508.01153
PDF
-
Scene Text Detection and Recognition 'in light of' Challenging Environmental Conditions using Aria Glasses Egocentric Vision Cameras
2025-07-22 08:12:00Joseph De Mathia, Carlos Francisco Moreno-Garc\'iaAbstract
In an era where wearable technology is reshaping applications, Scene Text Detection and Recognition (STDR) becomes a straightforward choice through the lens of egocentric vision. Leveraging Meta's Project Aria smart glasses, this paper investigates how environmental variables, such as lighting, distance, and resolution, affect the performance of state-of-the-art STDR algorithms in real-world scenarios. We introduce a novel, custom-built dataset captured under controlled conditions and evaluate two OCR pipelines: EAST with CRNN, and EAST with PyTesseract. Our findings reveal that resolution and distance significantly influence recognition accuracy, while lighting plays a less predictable role. Notably, image upscaling emerged as a key pre-processing technique, reducing Character Error Rate (CER) from 0.65 to 0.48. We further demonstrate the potential of integrating eye-gaze tracking to optimise processing efficiency by focusing on user attention zones. This work not only benchmarks STDR performance under realistic conditions but also lays the groundwork for adaptive, user-aware AR systems. Our contributions aim to inspire future research in robust, context-sensitive text recognition for assistive and research-oriented applications, such as asset inspection and nutrition analysis. The code is available at this https URL.
Abstract (translated)
在可穿戴技术重塑应用的时代，通过第一人称视角（egocentric vision）来看，场景文本检测和识别（STDR）成为了一个直接的选择。本文利用Meta的Project Aria智能眼镜，探讨了环境变量如光照、距离和分辨率如何影响先进STDR算法在现实世界中的性能表现。我们引入了一套新型的、专门构建的数据集，并在受控条件下对其进行评估，测试了两个OCR流程：EAST与CRNN相结合以及EAST与PyTesseract相结合的方法。 研究发现显示，分辨率和距离显著影响识别准确性，而光照的影响则较为不可预测。值得注意的是，图像放大作为一项关键预处理技术被证明可以降低字符错误率（CER）从0.65降至0.48。我们还展示了整合眼动追踪的潜力，通过关注用户的注意力区域来优化处理效率。 这项工作不仅在现实条件下对STDR性能进行了基准测试，而且还为自适应、用户感知增强现实系统奠定了基础。我们的贡献旨在激发未来研究，在辅助和科研导向应用（如资产检查与营养分析）中开发出更加稳健且符合上下文的文本识别技术。代码可在[此链接](https://example.com)获取。
URL
https://arxiv.org/abs/2507.16330
PDF
-
Aesthetics is Cheap, Show me the Text: An Empirical Evaluation of State-of-the-Art Generative Models for OCR
2025-07-20 18:43:09Peirong Zhang, Haowei Xu, Jiaxin Zhang, Guitao Xu, Xuhan Zheng, Zhenhua Yang, Junle Liu, Yuyi Zhang, Lianwen JinAbstract
Text image is a unique and crucial information medium that integrates visual aesthetics and linguistic semantics in modern e-society. Due to their subtlety and complexity, the generation of text images represents a challenging and evolving frontier in the image generation field. The recent surge of specialized image generators (\emph{e.g.}, Flux-series) and unified generative models (\emph{e.g.}, GPT-4o), which demonstrate exceptional fidelity, raises a natural question: can they master the intricacies of text image generation and editing? Motivated by this, we assess current state-of-the-art generative models' capabilities in terms of text image generation and editing. We incorporate various typical optical character recognition (OCR) tasks into our evaluation and broaden the concept of text-based generation tasks into OCR generative tasks. We select 33 representative tasks and categorize them into five categories: document, handwritten text, scene text, artistic text, and complex \& layout-rich text. For comprehensive evaluation, we examine six models across both closed-source and open-source domains, using tailored, high-quality image inputs and prompts. Through this evaluation, we draw crucial observations and identify the weaknesses of current generative models for OCR tasks. We argue that photorealistic text image generation and editing should be internalized as foundational skills into general-domain generative models, rather than being delegated to specialized solutions, and we hope this empirical analysis can provide valuable insights for the community to achieve this goal. This evaluation is online and will be continuously updated at our GitHub repository.
Abstract (translated)
文本图像是一种在现代电子社会中集视觉美感与语言语义于一体的独特且关键的信息媒介。由于其微妙性和复杂性，生成文本图像是图像生成领域中的一个充满挑战和不断发展的前沿方向。近年来，专业化的图像生成器（如Flux系列）以及统一的生成模型（如GPT-4o），展现出非凡的真实感效果，这引发了一个自然的问题：它们能否掌握文本图像生成与编辑的复杂性？受到这一问题的启发，我们评估了当前最先进的生成模型在文本图像生成和编辑方面的能力。我们在评估中纳入了一系列典型的光学字符识别(OCR)任务，并将基于文本的生成任务的概念扩展到了OCR生成任务。我们选择了33个具有代表性的任务，并将其分类为五类：文档、手写文本、场景文本、艺术文本以及复杂且布局丰富的文本。为了进行全面评估，我们在闭源和开源领域中选取了六种模型，并使用定制的高质量图像输入与提示语进行测试。通过这一评价过程，我们得出了关键性的观察结果并识别出当前OCR任务中的生成模型存在的弱点。我们认为，逼真的文本图像生成和编辑应当被融入到通用领域的生成模型之中，而不是委派给专门的解决方案来处理，并希望这种实证分析能够为社区实现此目标提供宝贵的见解。此次评估是在线进行且将持续在我们的GitHub仓库中更新。 这段文字介绍了对当前最先进的生成模型在文本图像生成和编辑方面的能力进行了全面评估的研究工作。研究不仅涵盖了多种类型的OCR任务，还提出了将逼真的文本图像生成和编辑技术融入通用领域生成模型的重要性，并分享了持续更新的实证分析结果。
URL
https://arxiv.org/abs/2507.15085
PDF
