Abstract
Online egocentric gaze estimation predicts where a camera wearer is looking from first-person video using only past and current frames, a task essential for augmented reality and assistive technologies. Unlike third-person gaze estimation, this setting lacks explicit head or eye signals, requiring models to infer current visual attention from sparse, indirect cues such as hand-object interactions and salient scene content. We observe that gaze exhibits strong temporal continuity during goal-directed activities: knowing where a person looked recently provides a powerful prior for predicting where they look next. Inspired by vision-conditioned autoregressive decoding in vision-language models, we propose ARGaze, which reformulates gaze estimation as sequential prediction: at each timestep, a transformer decoder predicts current gaze by conditioning on (i) current visual features and (ii) a fixed-length Gaze Context Window of recent gaze target estimates. This design enforces causality and enables bounded-resource streaming inference. We achieve state-of-the-art performance across multiple egocentric benchmarks under online evaluation, with extensive ablations validating that autoregressive modeling with bounded gaze history is critical for robust prediction. We will release our source code and pre-trained models.
Abstract (translated)
在线自拍视角估计是从第一人称视频中预测佩戴摄像头的人正在看向哪里，仅使用过去的和当前的帧来完成的任务。这一任务对于增强现实和辅助技术至关重要。与第三视角的视线估计不同，在这种情况下缺乏明确的头部或眼睛信号，模型需要从稀疏且间接的线索（如手部与物体的互动、场景中的显著内容）中推断当前的视觉注意点。 我们观察到，在目标导向活动中，注视表现出强烈的时间连续性：了解一个人最近看向哪里可以为预测他们下一步会看向哪里提供强有力的先验知识。受视觉条件下的自回归解码在视觉语言模型中的启发，我们提出了ARGaze方法，该方法将视线估计重新表述为序列预测问题：每一步，一个变压器解码器通过两个条件来预测当前的注视方向——（i）当前视觉特征和（ii）近期注视目标估计的一个固定长度的时间窗口（称为Gaze Context Window）。这种设计强制执行因果关系，并使有界资源下的流式推理成为可能。 在多个自拍基准测试中，我们实现了在线评估中的最佳性能，并通过广泛的消融实验验证了具有有限历史记录的自回归建模对于稳健预测的关键作用。我们将发布我们的源代码和预训练模型。
URL
https://arxiv.org/abs/2602.05132
