Abstract
Phoneme-based ASR factorizes recognition into speech-to-phoneme (S2P) and phoneme-to-grapheme (P2G), enabling cross-lingual acoustic sharing while keeping language-specific orthography in a separate module. While large language models (LLMs) are promising for P2G, multilingual P2G remains challenging due to language-aware generation and severe cross-language data imbalance. We study multilingual LLM-based P2G on the ten-language CV-Lang10 benchmark. We examine robustness strategies that account for S2P uncertainty, including DANP and Simplified SKM (S-SKM). S-SKM is a Monte Carlo approximation that avoids CTC-based S2P probability weighting in P2G training. Robust training and low-resource oversampling reduce the average WER from 10.56% to 7.66%.
Abstract (translated)
基于音素的自动语音识别（ASR）将识别过程分解为“语音转音素”（S2P）和“音素转字素”（P2G）两个模块，从而在保持语言特定正字法独立的同时实现跨语言声学共享。尽管大语言模型（LLMs）在P2G任务中展现出潜力，但多语言P2G仍面临语言感知生成与严重的跨语言数据不平衡两大挑战。本研究在十语言CV-Lang10基准上探讨基于多语言大模型的P2G方法，并检验了应对S2P不确定性的鲁棒性策略，包括DANP与简化版SKM（S-SKM）。S-SKM作为一种蒙特卡洛近似方法，避免了P2G训练中依赖CTC的S2P概率加权。通过鲁棒训练与低资源过采样策略，平均词错误率（WER）从10.56%降低至7.66%。
URL
https://arxiv.org/abs/2603.29217
