Abstract
We introduce Whisper-RIR-Mega, a benchmark dataset of paired clean and reverberant speech for evaluating automatic speech recognition (ASR) robustness to room acoustics. Each sample pairs a clean LibriSpeech utterance with the same utterance convolved with a real room impulse response from the RIR-Mega corpus, with stratified splits by reverberation time (RT60) and direct-to-reverberant ratio (DRR). We evaluate five Whisper models (tiny through large-v3) on 1600 test samples and report word error rate (WER) and character error rate (CER) under clean and reverberant conditions. Reverberation consistently degrades performance across all model sizes; the reverb penalty in WER ranges from 0.12 to 1.07 percentage points depending on the model. We release the dataset, evaluation code, and baseline results to support reproducible research on robust ASR.
Abstract (translated)
我们介绍了Whisper-RIR-Mega，这是一个用于评估自动语音识别（ASR）在房间声学条件下鲁棒性的配对纯净和回响语音基准数据集。每个样本包含一个来自LibriSpeech的纯净发音与该发音卷积后的实际房间脉冲响应（从RIR-Mega语料库中获取），并按照混响时间（RT60）和直达声到反射声比率（DRR）进行了分层划分。我们在1600个测试样本上评估了五个Whisper模型（tiny至large-v3），并在纯净和回响条件下报告单词错误率（WER）和字符错误率（CER）。混响效应在所有模型大小中都导致性能下降；根据模型的不同，WER的混响惩罚范围从0.12到1.07个百分点。我们发布了该数据集、评估代码及基线结果，以支持稳健ASR研究的可重复性。
URL
https://arxiv.org/abs/2603.02252
