Abstract
Eye tracking (ET) plays a critical role in augmented and virtual reality applications. However, rapidly deploying high-accuracy, on-device gaze estimation for new products remains challenging because hardware configurations (e.g., camera placement, camera pose, and illumination) often change across device generations. Visual foundation models (VFMs) are a promising direction for rapid training and deployment, and they excel on natural-image benchmarks; yet we find that off-the-shelf VFMs still struggle to achieve high accuracy on specialized near-eye infrared imagery. To address this gap, we introduce DistillGaze, a framework that distills a foundation model by leveraging labeled synthetic data and unlabeled real data for rapid and high-performance on-device gaze estimation. DistillGaze proceeds in two stages. First, we adapt a VFM into a domain-specialized teacher using self-supervised learning on labeled synthetic and unlabeled real images. Synthetic data provides scalable, high-quality gaze supervision, while unlabeled real data helps bridge the synthetic-to-real domain gap. Second, we train an on-device student using both teacher guidance and self-training. Evaluated on a large-scale, crowd-sourced dataset spanning over 2,000 participants, DistillGaze reduces median gaze error by 58.62% relative to synthetic-only baselines while maintaining a lightweight 256K-parameter model suitable for real-time on-device deployment. Overall, DistillGaze provides an efficient pathway for training and deploying ET models that adapt to hardware changes, and offers a recipe for combining synthetic supervision with unlabeled real data in on-device regression tasks.
Abstract (translated)
眼动追踪（ET）在增强现实与虚拟现实应用中发挥着关键作用。然而，由于硬件配置（如摄像头位置、姿态及光照条件）常随设备迭代而变化，快速为新产品部署高精度、端侧眼动估计仍是一项挑战。视觉基础模型（VFMs）为快速训练与部署提供了前景方向，其在自然图像基准测试中表现优异；但我们发现，现成VFMs在专业近眼红外图像上仍难以实现高精度。为弥合这一差距，我们提出了DistillGaze框架，该框架通过利用标注合成数据与无标注真实数据进行知识蒸馏，实现快速且高性能的端侧眼动估计。DistillGaze分两阶段进行：首先，利用标注合成图像与无标注真实图像进行自监督学习，将视觉基础模型适配为领域专用的教师模型——合成数据提供可扩展的高质量眼动监督，而无标注真实数据则有助于缩小合成与真实领域间的差异；其次，结合教师引导与自训练方法训练端侧学生模型。在涵盖2000多名参与者的大规模众包数据集上的评估表明，DistillGaze相较于仅使用合成数据的基线方法，将中位眼动误差降低了58.62%，同时保持了仅25.6万参数的轻量模型，适用于实时端侧部署。总体而言，DistillGaze为适应硬件变化的眼动追踪模型训练与部署提供了一条高效路径，并为端侧回归任务中结合合成监督与无标注真实数据提供了可行方案。
URL
https://arxiv.org/abs/2604.02509
