Abstract
Despite remarkable advances in natural language processing, developing effective systems for low-resource languages remains a formidable challenge, with performances typically lagging far behind high-resource counterparts due to data scarcity and insufficient linguistic resources. Cross-lingual knowledge transfer has emerged as a promising approach to address this challenge by leveraging resources from high-resource languages. In this paper, we investigate methods for transferring linguistic knowledge from high-resource languages to low-resource languages, where the number of labeled training instances is in hundreds. We focus on sentence-level and word-level tasks. We introduce a novel method, GETR (Graph-Enhanced Token Representation) for cross-lingual knowledge transfer along with two adopted baselines (a) augmentation in hidden layers and (b) token embedding transfer through token translation. Experimental results demonstrate that our GNN-based approach significantly outperforms existing multilingual and cross-lingual baseline methods, achieving 13 percentage point improvements on truly low-resource languages (Mizo, Khasi) for POS tagging, and 20 and 27 percentage point improvements in macro-F1 on simulated low-resource languages (Marathi, Bangla, Malayalam) across sentiment classification and NER tasks respectively. We also present a detailed analysis of the transfer mechanisms and identify key factors that contribute to successful knowledge transfer in this linguistic context.
Abstract (translated)
尽管自然语言处理领域取得了显著进展，但在低资源语言上开发有效的系统仍然是一项艰巨的挑战。由于数据稀缺和语言资源不足，这些系统的性能通常远远落后于高资源语言对应的系统。跨语言知识转移作为解决这一问题的一种有前景的方法已经浮现，通过这种方法可以利用来自高资源语言的数据来帮助低资源语言。 在本文中，我们研究了将语言知识从高资源语言转移到低资源语言的方法，其中标签训练实例的数量只有几百个。我们的重点是句子级和词汇级的任务。为此，我们提出了一种名为GETR（图增强标记表示）的新方法来进行跨语言知识转移，并采用两种基线方法：(a) 隐藏层中的数据扩充和 (b) 通过词元翻译进行词元嵌入传递。 实验结果表明，我们的基于GNN的方法在多语言和跨语言基准方法上取得了显著的改进，在真正低资源的语言（米佐语、卡西语）上的词性标注任务中，性能提升了13个百分点。对于模拟的低资源语言（马拉地语、孟加拉语、马拉雅拉姆语），我们在情感分类和命名实体识别（NER）任务上分别实现了20和27个百分点的宏观F1得分提升。 此外，我们还对转移机制进行了详细的分析，并确定了在这种语言环境中成功实现知识传递的关键因素。
URL
https://arxiv.org/abs/2602.05599
