Abstract
Pre-trained gaze models learn to identify useful patterns commonly found across users, but subtle user-specific variations (i.e., eyelid shape or facial structure) can degrade model performance. Test-time personalization (TTP) adapts pre-trained models to these user-specific domain shifts using only a few unlabeled samples. Efficient fine-tuning is critical in performing this domain adaptation: data and computation resources can be limited-especially for on-device customization. While popular parameter-efficient fine-tuning (PEFT) methods address adaptation costs by updating only a small set of weights, they may not be taking full advantage of structures encoded in pre-trained filters. To more effectively leverage existing structures learned during pre-training, we reframe personalization as a process to reweight existing features rather than learning entirely new ones. We present Attentive Low-Rank Filter Adaptation (Alfa) to adapt gaze models by reweighting semantic patterns in pre-trained filters. With Alfa, singular value decomposition (SVD) extracts dominant spatial components that capture eye and facial characteristics across users. Via an attention mechanism, we need only a few unlabeled samples to adjust and reweight pre-trained structures, selectively amplifying those relevant to a target user. Alfa achieves the lowest average gaze errors across four cross-dataset gaze benchmarks, outperforming existing TTP methods and low-rank adaptation (LoRA)-based variants. We also show that Alfa's attentive low-rank methods can be applied to applications beyond vision, such as diffusion-based language models.
Abstract (translated)
预训练的眼动模型能够学习识别跨用户常用的有用模式，但细微的个体差异（如眼睑形状或面部结构）可能会降低模型性能。测试时间个性化（TTP）通过少量未标记样本使预训练模型适应这些个体特定领域变化。高效的微调对于执行这种域适应至关重要：数据和计算资源可能有限，特别是在设备上的定制时更为明显。虽然流行的参数高效微调（PEFT）方法通过仅更新一组小权重来解决适应成本问题，但它们可能未能充分利用预先训练的过滤器中编码的结构信息。为了更有效地利用预训练期间学习到的现有结构，我们将个性化重新定义为一个重新加权现有特征的过程，而不是完全学习新的特征。 我们提出了注意力低秩滤波器调整（Alfa），通过重新加权预训练过滤器中的语义模式来适应眼动模型。借助Alfa，奇异值分解(SVD)提取了主要的空间成分，这些成分捕捉到跨用户的眼部和面部特性。通过一个注意机制，在仅使用少量未标记样本的情况下，我们就可以调整并重新加权预先训练的结构，并且选择性地放大与目标用户相关的部分。 在四个跨数据集眼动基准测试中，Alfa实现了最低的平均眼动误差，优于现有的TTP方法和低秩适应（LoRA）变体。此外，我们也展示了Alfa的注意力低秩方法可以应用于视觉以外的应用程序，例如基于扩散的语言模型。
URL
https://arxiv.org/abs/2603.08445
