Abstract
Adversarial perturbations exploit vulnerabilities in automatic speech recognition (ASR) systems while preserving human perceived linguistic content. Neural audio codecs impose a discrete bottleneck that can suppress fine-grained signal variations associated with adversarial noise. We examine how the granularity of this bottleneck, controlled by residual vector quantization (RVQ) depth, shapes adversarial robustness. We observe a non-monotonic trade-off under gradient-based attacks: shallow quantization suppresses adversarial perturbations but degrades speech content, while deeper quantization preserves both content and perturbations. Intermediate depths balance these effects and minimize transcription error. We further show that adversarially induced changes in discrete codebook tokens strongly correlate with transcription error. These gains persist under adaptive attacks, where neural codec configurations outperform traditional compression defenses.
Abstract (translated)
对抗性扰动在利用自动语音识别（ASR）系统的漏洞的同时，保持人类感知的语言内容不变。神经音频编解码器施加了一个离散瓶颈，可以抑制与对抗噪声相关的细微信号变化。我们研究了这个瓶颈的粒度如何通过残差向量量化（RVQ）深度来控制，并探讨这对对抗鲁棒性的影响。我们在基于梯度的攻击下观察到一种非单调权衡：浅层量化可以抑制对抗性扰动，但会降低语音内容的质量；而深层量化则既能保留内容又能保留扰动。中间深度则能平衡这些效果并最小化转录错误。此外，我们还发现由对抗攻击引起的离散代码本令牌的变化与转录误差有很强的相关性。这种优势在自适应攻击下依然存在，其中神经编解码器配置优于传统的压缩防御措施。
URL
https://arxiv.org/abs/2603.09034
