Abstract
Reinforcement learning based post-training paradigms for Video Large Language Models (VideoLLMs) have achieved significant success by optimizing for visual-semantic tasks such as captioning or VideoQA. However, while these approaches effectively enhance perception abilities, they primarily target holistic content understanding, often lacking explicit supervision for intrinsic temporal coherence and inter-frame correlations. This tendency limits the models' ability to capture intricate dynamics and fine-grained visual causality. To explicitly bridge this gap, we propose a novel post-training objective: Masked Video Prediction (MVP). By requiring the model to reconstruct a masked continuous segment from a set of challenging distractors, MVP forces the model to attend to the sequential logic and temporal context of events. To support scalable training, we introduce a scalable data synthesis pipeline capable of transforming arbitrary video corpora into MVP training samples, and further employ Group Relative Policy Optimization (GRPO) with a fine-grained reward function to enhance the model's understanding of video context and temporal properties. Comprehensive evaluations demonstrate that MVP enhances video reasoning capabilities by directly reinforcing temporal reasoning and causal understanding.
Abstract (translated)
基于强化学习的视频大型语言模型（VideoLLM）在视觉语义任务如描述和视频问答中的后训练范式取得了显著成功。然而，尽管这些方法有效提高了感知能力，并且主要针对整体内容理解进行了优化，它们通常缺乏对内在时间连贯性和帧间相关性的显式监督。这种倾向限制了模型捕捉复杂动态变化和细粒度视觉因果关系的能力。 为了明确填补这一缺口，我们提出了一种新的后训练目标：遮罩视频预测（Masked Video Prediction, MVP）。通过要求模型从一系列具有挑战性的干扰项中重构一个被遮盖的连续片段，MVP迫使模型关注事件的顺序逻辑和时间背景。为支持可扩展性训练，我们引入了一个能够将任意视频语料库转换成用于MVP训练样本的数据合成流水线，并进一步采用细粒度奖励函数的分组相对策略优化（Group Relative Policy Optimization, GRPO）来增强模型对视频背景的理解及时间属性。 全面的评估表明，通过直接强化时间推理和因果理解能力，MVP提升了视频推理的能力。
URL
https://arxiv.org/abs/2601.03781
