Abstract
We present a new and accurate approach for gaze estimation on consumer computing devices. We take advantage of continued strides in the quality of user-facing cameras found in e.g., smartphones, laptops, and desktops - 4K or greater in high-end devices - such that it is now possible to capture the 2D reflection of a device's screen in the user's eyes. This alone is insufficient for accurate gaze tracking due to the near-infinite variety of screen content. Crucially, however, the device knows what is being displayed on its own screen - in this work, we show this information allows for robust segmentation of the reflection, the location and size of which encodes the user's screen-relative gaze target. We explore several strategies to leverage this useful signal, quantifying performance in a user study. Our best performing model reduces mean tracking error by ~8% compared to a baseline appearance-based model. A supplemental study reveals an additional 10-20% improvement if the gaze-tracking camera is located at the bottom of the device.
Abstract (translated)
我们提出了一种在消费级计算设备上进行视线估计的新且精确的方法。我们利用了面向用户的摄像头（例如智能手机、笔记本电脑和台式机中的摄像头）在质量上的持续进步——高端设备中已达到4K或更高分辨率——使得现在可以捕捉到用户眼中设备屏幕的二维反射像。然而仅凭这一点还不足以实现精确的视线追踪，因为屏幕内容近乎无限多样。然而关键在于，设备自身知道屏幕上正在显示的内容——在本研究中，我们证明这一信息能够实现反射像的鲁棒分割，而反射像的位置和尺寸编码了用户相对于屏幕的注视目标。我们探索了多种利用这一有效信号的策略，并通过用户研究量化了性能。我们表现最佳的模型相比基线基于外观的模型，将平均追踪误差降低了约8%。一项补充研究揭示，若视线追踪摄像头位于设备底部，还可实现额外10-20%的改进。
URL
https://arxiv.org/abs/2603.19588
