Abstract
Despite recent progress on the short-video Text-Visual Question Answering (ViteVQA) task - largely driven by benchmarks such as M4-ViteVQA - existing datasets still suffer from limited video duration and narrow evaluation scopes, making it difficult to adequately assess the growing capabilities of powerful multimodal large language models (MLLMs). To address these limitations, we introduce TextVidBench, the first benchmark specifically designed for long-video text question answering (>3 minutes). TextVidBench makes three key contributions: 1) Cross-domain long-video coverage: Spanning 9 categories (e.g., news, sports, gaming), with an average video length of 2306 seconds, enabling more realistic evaluation of long-video understanding. 2) A three-stage evaluation framework: "Text Needle-in-Haystack -> Temporal Grounding -> Text Dynamics Captioning". 3) High-quality fine-grained annotations: Containing over 5,000 question-answer pairs with detailed semantic labeling. Furthermore, we propose an efficient paradigm for improving large models through: (i) introducing the IT-Rope mechanism and temporal prompt engineering to enhance temporal perception, (ii) adopting non-uniform positional encoding to better handle long video sequences, and (iii) applying lightweight fine-tuning on video-text data. Extensive experiments on multiple public datasets as well as TextVidBench demonstrate that our new benchmark presents significant challenges to existing models, while our proposed method offers valuable insights into improving long-video scene text understanding capabilities.
Abstract (translated)
尽管在短视频文本-视觉问答（ViteVQA）任务上取得了近期进展——这主要得益于M4-ViteVQA等基准测试的推动，现有的数据集仍然存在视频时长有限和评估范围狭窄的问题，使得难以充分评价强大多模态大型语言模型（MLLMs）的能力。为了解决这些问题，我们引入了TextVidBench，这是第一个专门针对长时间视频文本问答（>3分钟）设计的基准。 TextVidBench做出了三项关键贡献： 1. **跨域长视频覆盖**：涵盖9个类别（例如新闻、体育、游戏），平均视频长度为2306秒，这使得对长视频理解的评估更加现实。 2. **三阶段评估框架**：“文本大海捞针 -> 时间定位 -> 文本动态描述”。 3. **高质量细粒度注释**：包含超过5,000个问题-答案配对，并带有详细的语义标签。 此外，我们还提出了一种通过以下方式改进大规模模型的高效范式： (i) 引入IT-Rope机制和时间提示工程来增强时间感知能力； (ii) 采用非均匀位置编码以更好地处理长视频序列； (iii) 在视频文本数据上进行轻量级微调。 在多个公共数据集以及TextVidBench上的广泛实验表明，我们的新基准对现有模型提出了重大挑战，而我们提出的改进方法为提升长时间视频场景中的文本理解能力提供了有价值的见解。
URL
https://arxiv.org/abs/2506.04983
