Abstract
Automated animal identification is a practical task for reuniting lost pets with their owners, yet current systems often struggle due to limited dataset scale and reliance on unimodal visual cues. This study introduces a multimodal verification framework that enhances visual features with semantic identity priors derived from synthetic textual descriptions. We constructed a massive training corpus of 1.9 million photographs covering 695,091~unique animals to support this investigation. Through systematic ablation studies, we identified SigLIP2-Giant and E5-Small-v2 as the optimal vision and text backbones. We further evaluated fusion strategies ranging from simple concatenation to adaptive gating to determine the best method for integrating these modalities. Our proposed approach utilizes a gated fusion mechanism and achieved a Top-1 accuracy of 84.28\% and an Equal Error Rate of 0.0422 on a comprehensive test protocol. These results represent an 11\% improvement over leading unimodal baselines and demonstrate that integrating synthesized semantic descriptions significantly refines decision boundaries in large-scale pet re-identification.
Abstract (translated)
自动化动物识别是一项实用的任务，有助于将走失的宠物归还给主人。然而，目前的系统往往因数据集规模有限和依赖单一视觉线索而难以实现这一目标。本研究引入了一种多模态验证框架，该框架通过合成文本描述中提取的语义身份先验来增强视觉特征。我们构建了一个包含190万张照片的数据集，覆盖了695,091只独特动物，以支持这项研究。 通过对多种实验进行系统性分析，我们确定SigLIP2-Giant和E5-Small-v2是最佳的视觉和文本骨干网络。此外，我们还评估了一系列融合策略，从简单的拼接到自适应门控机制，以确定将这些模式整合的最佳方法。我们的提议方法使用了门控融合机制，并在全面测试协议下实现了84.28%的Top-1准确率和0.0422的等错误率。 与领先的单模态基线相比，这一结果提高了11%，证明了将合成语义描述整合到大规模宠物再识别任务中可以显著优化决策边界。
URL
https://arxiv.org/abs/2603.02270
