Abstract
Large language models (LLMs) have driven substantial advances in speech language models (SpeechLMs), yielding strong performance in automatic speech recognition (ASR) under high-resource conditions. However, existing benchmarks predominantly focus on high-resource languages, leaving the ASR behavior of SpeechLMs in low-resource languages insufficiently understood. This gap is critical, as practical ASR systems must reliably support low-resource languages and generalize across diverse language families, and it directly hinders the deployment of SpeechLM-based ASR in real-world multilingual scenarios. As a result, it is essential to evaluate SpeechLMs on low-resource languages to ensure their generalizability across different language families. To address this problem, we propose \textbf{LoASR-Bench}, a comprehensive benchmark designed to evaluate \textbf{lo}w-resource \textbf{a}utomatic \textbf{s}peech \textbf{r}ecognition (\textbf{ASR}) of the latest SpeechLMs across diverse language families. LoASR-Bench comprises 25 languages from 9 language families, featuring both Latin and non-Latin scripts, enabling cross-linguistic and cross-script assessment of ASR performance of current SpeechLMs. Experimental results highlight the limitations of the latest SpeechLMs in handling real-world low-resource languages.
Abstract (translated)
大语言模型（LLMs）推动了语音语言模型（SpeechLMs）的显著进步，在高资源条件下自动语音识别（ASR）任务中展现出强劲性能。然而，现有基准测试主要关注高资源语言，导致SpeechLMs在低资源语言中的ASR行为尚未得到充分理解。这一空白至关重要，因为实际ASR系统必须可靠地支持低资源语言，并跨不同语系实现泛化，这直接阻碍了基于SpeechLM的ASR在真实多语言场景中的部署。因此，为评估SpeechLMs在不同语系间的泛化能力，必须在低资源语言上进行测试。针对这一问题，我们提出了\textbf{LoASR-Bench}，这是一个用于评估最新SpeechLMs在跨语系\textbf{低资源自动语音识别（ASR）}表现的综合基准。LoASR-Bench涵盖9个语系的25种语言，包含拉丁与非拉丁文字系统，可对当前SpeechLMs的ASR性能进行跨语言与跨文字评估。实验结果表明，最新SpeechLMs在处理真实低资源语言时存在明显局限。
URL
https://arxiv.org/abs/2603.20042
