Abstract
Although the deep integration of the Automatic Speech Recognition (ASR) system with Large Language Models (LLMs) has significantly improved accuracy, the deployment of such systems in low-latency streaming scenarios remains challenging. In this paper, we propose Uni-ASR, a unified framework based on LLMs that integrates both non-streaming and streaming speech recognition capabilities. We propose a joint training paradigm that enables the system to seamlessly transition between two recognition modes without any architectural modifications. Furthermore, we introduce a context-aware training paradigm and a co-designed fallback decoding strategy, which can enhance streaming recognition accuracy without introducing additional latency. The experimental results demonstrate that Uni-ASR not only achieves competitive performance within non-streaming mode, but also demonstrates strong effectiveness in streaming scenarios under diverse latency constraints.
Abstract (translated)
尽管自动语音识别（ASR）系统与大型语言模型（LLM）的深度集成显著提高了准确性，但在低延迟流式场景中部署此类系统仍然具有挑战性。在本文中，我们提出了Uni-ASR，这是一种基于LLM的统一框架，集成了非流式和流式语音识别能力。我们提出了一种联合训练范式，使系统能够在两种识别模式之间无缝切换，而无需进行架构修改。此外，我们引入了上下文感知训练范式以及协同设计的退避解码策略，这可以在不增加额外延迟的情况下提高流式识别准确性。实验结果表明，Uni-ASR不仅在非流式模式下实现了竞争性性能，还在不同延迟约束条件下的流式场景中表现出强大的有效性。
URL
https://arxiv.org/abs/2603.11123
