Abstract
Achieving a balance between high-fidelity visual quality and low-latency streaming remains a formidable challenge in audio-driven portrait generation. Existing large-scale models often suffer from prohibitive computational costs, while lightweight alternatives typically compromise on holistic facial representations and temporal stability. In this paper, we propose SoulX-FlashHead, a unified 1.3B-parameter framework designed for real-time, infinite-length, and high-fidelity streaming video generation. To address the instability of audio features in streaming scenarios, we introduce Streaming-Aware Spatiotemporal Pre-training equipped with a Temporal Audio Context Cache mechanism, which ensures robust feature extraction from short audio fragments. Furthermore, to mitigate the error accumulation and identity drift inherent in long-sequence autoregressive generation, we propose Oracle-Guided Bidirectional Distillation, leveraging ground-truth motion priors to provide precise physical guidance. We also present VividHead, a large-scale, high-quality dataset containing 782 hours of strictly aligned footage to support robust training. Extensive experiments demonstrate that SoulX-FlashHead achieves state-of-the-art performance on HDTF and VFHQ benchmarks. Notably, our Lite variant achieves an inference speed of 96 FPS on a single NVIDIA RTX 4090, facilitating ultra-fast interaction without sacrificing visual coherence.
Abstract (translated)
在音频驱动的肖像生成领域，实现高保真视觉质量和低延迟流媒体之间的平衡仍然是一个重大挑战。现有的大规模模型通常面临计算成本过高的问题，而轻量级替代方案则往往牺牲整体面部表示和时间稳定性。为此，本文提出了SoulX-FlashHead，这是一种统一的13亿参数框架，旨在实现实时、无限长度且高保真的流媒体视频生成。 为了应对流媒体场景中音频特征不稳定的问题，我们引入了“流感知时空预训练”（Streaming-Aware Spatiotemporal Pre-training），并配备了一个时间音频上下文缓存机制(Temporal Audio Context Cache)，以确保从短音频片段中提取稳健的特征。此外，为了解决长时间序列自回归生成过程中固有的错误累积和身份漂移问题，我们提出了Oracle-Guided双向蒸馏(Oracle-Guided Bidirectional Distillation)方法，利用地面真实运动先验提供精确的物理引导。 为了支持稳健训练，我们还推出了VividHead，这是一个大规模高质量的数据集，包含782小时严格对齐的视频片段。经过广泛的实验验证，SoulX-FlashHead在HDTF和VFHQ基准测试中实现了最先进的性能表现。值得一提的是，我们的Lite变体能够在单个NVIDIA RTX 4090上实现每秒96帧（FPS）的推理速度，从而支持超快交互的同时不牺牲视觉连贯性。
URL
https://arxiv.org/abs/2602.07449
