Abstract
In speech evaluation, an Automatic Speech Recognition (ASR) model often computes time boundaries and phoneme posteriors for input features. However, limited data for ASR training hinders expansion of speech evaluation to low-resource languages. Open-source weakly-supervised models are capable of ASR over many languages, but they are frame-asynchronous and not phonemic, hindering feature extraction for speech evaluation. This paper proposes to overcome incompatibilities for feature extraction with weakly-supervised models, easing expansion of speech evaluation to low-resource languages. Phoneme posteriors are computed by mapping ASR hypotheses to a phoneme confusion network. Word instead of phoneme-level speaking rate and duration are used. Phoneme and frame-level features are combined using a cross-attention architecture, obviating phoneme time alignment. This performs comparably with standard frame-synchronous features on English speechocean762 and low-resource Tamil datasets.
Abstract (translated)
在语音评估中，自动语音识别（ASR）模型通常会对输入特征计算时间边界和音素后验概率。然而，ASR训练数据有限阻碍了语音评估向低资源语言的扩展。开源弱监督模型虽能支持多语言ASR，但其帧异步且非音素级的特性阻碍了语音评估的特征提取。本文提出克服弱监督模型在特征提取上的不兼容性，以促进语音评估向低资源语言扩展。通过将ASR假设映射至音素混淆网络来计算音素后验概率，并采用词级而非音素级的语速和时长特征。利用交叉注意力架构融合音素与帧级特征，无需音素时间对齐。该方法在英语speechocean762和低资源泰米尔语数据集上的表现与标准帧同步特征相当。
URL
https://arxiv.org/abs/2603.25150
