Abstract
Recent advances in automatic speech recognition (ASR) and speech enhancement have led to a widespread assumption that improving perceptual audio quality should directly benefit recognition accuracy. In this work, we rigorously examine whether this assumption holds for modern zero-shot ASR systems. We present a systematic empirical study on the impact of Segment Anything Model Audio by Meta AI, a recent foundation-scale speech enhancement model proposed by Meta, when used as a preprocessing step for zero-shot transcription with Whisper. Experiments are conducted across multiple Whisper model variants and two linguistically distinct noisy speech datasets: a real-world Bengali YouTube corpus and a publicly available English noisy dataset. Contrary to common intuition, our results show that SAM-Audio preprocessing consistently degrades ASR performance, increasing both Word Error Rate (WER) and Character Error Rate (CER) compared to raw noisy speech, despite substantial improvements in signal-level quality. Objective Peak Signal-to-Noise Ratio analysis on the English dataset confirms that SAM-Audio produces acoustically cleaner signals, yet this improvement fails to translate into recognition gains. Therefore, we conducted a detailed utterance-level analysis to understand this counterintuitive result. We found that the recognition degradation is a systematic issue affecting the majority of the audio, not just isolated outliers, and that the errors worsen as the Whisper model size increases. These findings expose a fundamental mismatch: audio that is perceptually cleaner to human listeners is not necessarily robust for machine recognition. This highlights the risk of blindly applying state-of-the-art denoising as a preprocessing step in zero-shot ASR pipelines.
Abstract (translated)
近期在自动语音识别（ASR）和语音增强领域的进展导致了一种广泛的看法，即提高感知音频质量应直接有助于提升识别准确性。然而，在这项工作中，我们严谨地检验了这种假设是否适用于现代的零样本ASR系统。我们对Meta AI提出的最近一种大规模基础模型——Segment Anything Model Audio (SAM-Audio)进行了一项系统的实证研究，该模型用作使用Whisper进行零样本转录的预处理步骤时的影响。实验在多个Whisper模型变体以及两种语言不同的嘈杂语音数据集上进行：一个是现实世界的孟加拉语YouTube语料库，另一个是公开可用的英语嘈杂数据集。 与常见的直觉相反，我们的结果显示SAM-Audio预处理一致地降低了ASR性能，相比于原始的嘈杂音频，无论是词错误率（WER）还是字符错误率（CER）都有所增加。尽管信号级质量有了显著提升，但这种改进并未转化为识别效果上的增益。对英语数据集进行客观峰值信噪比分析表明，SAM-Audio确实产生了音质更清晰的信号，然而这一改善未能转化成实际的识别收益。 因此，我们进行了详细的句子级别分析以理解这一反直觉的结果。发现识别性能下降是一个影响大部分音频的系统性问题，而不仅仅是孤立的异常值，并且随着Whisper模型规模的增长，错误率也随之增加。这些发现在根本上揭示了这样一种矛盾：对于人类听者来说更为清晰的音频不一定是机器识别时更加稳健的信号。 这项研究强调了在零样本ASR流程中盲目使用最先进的降噪技术作为预处理步骤的风险，并提出了挑战当前假设的重要见解，即优化感知音质并不一定意味着能够直接提升语音识别性能。
URL
https://arxiv.org/abs/2603.04710
