NetEase Youdao releases Confucius4-R2T2, a true streaming speech recognition model built on Qwen3-ASR with 200-600 ms latency and append-only output.
Confucius4-R2T2 ('Real Real-Time Transcription') is a streaming automatic speech recognition model from NetEase Youdao built on Qwen3-ASR, featuring configurable decoding chunks from 80 ms to 2 s and an append-only output mode that commits transcript text permanently without revisions. It reports 200-600 ms average latency with accuracy close to offline recognition, and claims state-of-the-art latency and recognition quality among open-source models while remaining competitive with leading closed-source systems. The model is optimized for Chinese and English, supports context and hotword prompts, and ships with vLLM and Hugging Face transformers backends plus WebSocket streaming inference code under a NetEase model license and Apache 2.0 code license.
Streaming ASR with 200-600 ms average latency and near-offline accuracy
Configurable decoding chunks from 80 ms to 2 seconds for latency/accuracy trade-offs
Append-only output commits text without revisions, suited to live captioning and LLM agents
Built on Qwen3-ASR with vLLM and transformers backends, WebSocket server included
Claims SOTA among open-source models; optimized for Chinese and English
<a href="https://raw.githubusercontent.com/netease-youdao/Confucius4-R2T2/refs/heads/master/MODEL_LICENSE"><img src="https://img.shields.io/badge/model_license-NetEase-blue" alt="Model license: NetEase Model Use License Agreement"></a>
Confucius4-R2T2 is a low-latency and high-accuracy true streaming Automatic Speech Recognition (ASR) model that features fine-grained and configurable decoding chunks from 80 ms to 2 s. The model operates in append-only output mode: committing transcript text permanently without revising previous words, which is critical for applications where text must be processed or acted upon instantly. This results in a smoother user experience, avoiding disruptive text revisions and visual flickering in real-time applications, such as Real-Time Live Captioning & Subtitling, Downstream NLP Pipelines & LLM Agents, Simultaneous Speech Translation, etc.
R2T2, short for Real Real-Time Transcription, is built upon the Qwen3-ASR model. And it is trained with a unique set of data construction techniques including stable-prefix data, forced time-alignment data, and token-level audio segmentation. Combined with a Longest Stable Prefix (LSP) learning paradigm (tech report will be released soon), R2T2 can dynamically determine when a stable prefix can be safely emitted and when additional audio context is needed. By exposing only stable prefixes, the model provides high-quality context that conditions subsequent predictions while guaranteeing that previously emitted text remains unchanged. Despite its streaming design, R2T2 maintains strong accuracy in offline recognition.
- **Low-latency and high accuracy streaming recognition** — The model achieves accuracy close to that of offline recognition, with only 200 to 600 milliseconds average latency.
- **Stable streaming output** — Emitted text is committed as it arrives and remains unchanged.
- **Configurable low-latency chunking** - Supports decoding chunks from 80 ms to 2 s for different latency/accuracy trade-offs.
- **No loss in offline accuracy** — Adding streaming support does not degrade offline recognition accuracy.
- **vLLM backend** — Provides high-throughput inference. A Hugging Face `transformers` backend is also available.
- **Context and hotword prompts** — Natively supported.
- **Multilingual support** — Optimized for **Chinese and English**, while also supporting a broad range of additional languages.
Experimental results show that R2T2 achieves state-of-the-art (SOTA) performance in both latency and recognition quality among a range of open-source models, while remaining competitive with leading closed-source systems. The [GitHub repository](https://github.com/netease-youdao/Confucius4-R2T2) provides inference code, a minimal usage example, and a vLLM-based backend supporting both offline and real-time streaming inference.
## Table of Contents
- [Overview](#overview)
- [Demo](#demo)
- [Side-by-side comparison with GPT-Live-Transcribe](#side-by-side-comparison-with-gpt-live-transcribe)
- [Additional resources](#additional-resources)
- [Evaluation](#evaluation)
- [Streaming performance](#streaming-performance)
- [Accuracy](#accuracy)
- [English](#english)
- [Chinese](#chinese)
- [Installation](#installation)
- [Clone the repository](#clone-the-repository)
- [Option 1: Conda](#option-1-conda)
- [Option 2: uv](#option-2-uv)
- [Docker (recommended)](#docker-recommended)
- [1. Start a container](#1-start-a-container)
- [2. Run the example inside the container](#2-run-the-example-inside-the-container)
- [3. Manage the container](#3-manage-the-container)
<p><i>Figure 1. Overall framework of R2T2.</i></p>
</div>
## Demo
### Side-by-side comparison with GPT-Live-Transcribe
<div align="center">
<video controls playsinline preload="metadata" width="90%" src="https://github.com/user-attachments/assets/1b21c04a-766a-434f-96dc-580376b305f1" title="GPT-Live-Transcribe and R2T2 processing the same audio together in real time — a side-by-side comparison.">
Your browser does not support embedded video.
</video>
<p><a href="https://github.com/user-attachments/assets/1b21c04a-766a-434f-96dc-580376b305f1">Watch the comparison video</a></p>
<p><i>Figure 2. GPT-Live-Transcribe and R2T2 processing the same audio, shown together in real time — a side-by-side comparison.</i></p>
</div>
### Additional resources
More demonstrations, comparisons, and supporting resources will be added here.
## Evaluation
> If you are an author or maintainer of a model included in these comparisons and have questions or concerns about the results, please feel free to contact us through the [GitHub issue tracker](https://github.com/netease-youdao/Confucius4-R2T2/issues). We are happy to share evaluation details and work with you to verify or correct them.
### Streaming performance
The streaming API supports decoding chunks from 80 ms to 2 s; the figures below show representative WER/latency trade-offs at 160 ms.
<div align="center">
<img src="https://raw.githubusercontent.com/netease-youdao/Confucius4-R2T2/refs/heads/master/resources/asr_en_wer_latency.svg" alt="English WER and retrospective chunk-wise latency comparison across ASR models and configurations" width="80%">
<p><i>Figure 3. English WER and retrospective chunk-wise latency across model and configuration settings.</i></p>
</div>
<div align="center">
<img src="https://raw.githubusercontent.com/netease-youdao/Confucius4-R2T2/refs/heads/master/resources/asr_cn_wer_latency.svg" alt="Chinese CER and retrospective chunk-wise latency comparison across ASR models and configurations" width="80%">
<p><i>Figure 4. Chinese CER and retrospective chunk-wise latency across model and configuration settings.</i></p>
</div>
<div align="center">
<img src="https://raw.githubusercontent.com/netease-youdao/Confucius4-R2T2/refs/heads/master/resources/asr_pareto_wer_latency.svg" alt="English and Chinese accuracy-latency Pareto frontier for representative streaming ASR configurations" width="96%">
<p><i>Figure 5. Accuracy-latency Pareto frontier. Lower-left is better; the frontier uses retrospective chunk-wise mean fuzzy latency.</i></p>
</div>
### Accuracy
English results use WER (%), and Chinese results use CER (%); lower is better.
※ Pseudo-streaming model: its partial transcript may revise previously emitted text; unmarked models use true streaming, append-only output.