Abstract
Gaze estimation is instrumental in modern virtual reality (VR) systems. Despite significant progress in remote-camera gaze estimation, VR gaze research remains constrained by data scarcity - particularly the lack of large-scale, accurately labeled datasets captured with the off-axis camera configurations typical of modern headsets. Gaze annotation is difficult since fixation on intended targets cannot be guaranteed. To address these challenges, we introduce VRGaze - the first large-scale off-axis gaze estimation dataset for VR - comprising 2.1 million near-eye infrared images collected from 68 participants. We further propose GazeShift, an attention-guided unsupervised framework for learning gaze representations without labeled data. Unlike prior redirection-based methods that rely on multi-view or 3D geometry, GazeShift is tailored to near-eye infrared imagery, achieving effective gaze-appearance disentanglement in a compact, real-time model. GazeShift embeddings can be optionally adapted to individual users via lightweight few-shot calibration, achieving a 1.84-degree mean error on VRGaze. On the remote-camera MPIIGaze dataset, the model achieves a 7.15-degree person-agnostic error, doing so with 10x fewer parameters and 35x fewer FLOPs than baseline methods. Deployed natively on a VR headset GPU, inference takes only 5 ms. Combined with demonstrated robustness to illumination changes, these results highlight GazeShift as a label-efficient, real-time solution for VR gaze tracking. Project code and the VRGaze dataset are released at this https URL.
Abstract (translated)
注视估计在现代虚拟现实（VR）系统中至关重要。尽管远程摄像头注视估计方面已取得重大进展，但VR注视研究仍然受限于数据稀缺问题——特别是缺乏大规模、准确标注的数据集，这些数据集使用的是现代头显典型的非轴心相机配置所捕获的图像。由于难以保证用户能够固定在预设的目标上，因此注释工作也变得困难重重。为了解决这些问题，我们推出了VRGaze——首个用于VR的大规模非轴心注视估计数据集，该数据集中包含从68名参与者收集的210万张近眼红外图像。此外，我们还提出了GazeShift框架，这是一种注意力引导型无监督学习方法，可以在没有标注数据的情况下学习注视表示形式。不同于依赖于多视角或三维几何原理的传统重定向方法，GazeShift专门针对近眼红外成像技术进行了优化，在紧凑的实时模型中实现了有效的注视-外观解耦。通过轻量级的少量样本校准，GazeShift嵌入式可以个性化适应各个用户，使其在VRGaze数据集上的平均误差仅为1.84度。在远程摄像头MPIIGaze数据集上，该模型也表现出色，其对个人身份无依赖的错误率为7.15度，而使用的参数量和计算量（FLOPs）分别是基线方法的十分之一和三十五分之一。当直接部署到VR头显GPU上时，推理过程仅需花费5毫秒的时间。结合其在光照变化下的鲁棒性表现来看，这些结果突显了GazeShift作为一种标签效率高、实时性的解决方案，在VR注视追踪中的优势。项目代码及VRGaze数据集可在提供的网址上获取。
URL
https://arxiv.org/abs/2603.07832
