Edge0/Audio8-ASR-Infinite

TL;DR: Audio8-ASR-Infinite is a ~7B bilingual (zh/en) streaming ASR model that emits one text token per fixed audio clock tick (80/120/160 ms), following the Voxtral-realtime / delayed-streams recipe, and pairs it with a rolling 30-second KV cache with exact RoPE re-basing so it can transcribe indefinitely at constant memory and latency. Self-reported numbers show it crushing the compared streaming baselines on Chinese (CER 1.75 vs 12.9–16.8 on AISHELL-1 at ~480 ms delay) while staying competitive on LibriSpeech — worth testing if you build realtime voice systems, with the usual caveats about self-reported evals from a new org.

What it is and why it's interesting

Most "streaming" ASR you meet in practice is an offline model behind a chunking wrapper, with all the drift, boundary artifacts, and unbounded state that implies. This is a native streaming architecture: the decoder makes a decision every clock tick, latency is a first-class, configurable quantity (240–560 ms, expressed as a number of delay tokens), and long-running operation is handled architecturally rather than by resetting state. The rolling-KV-with-RoPE-rebasing trick is what makes the "24/7 without drifting" claim plausible: the checkpoint's native context is only 30 s, but the window slides continuously with positions re-based exactly, so memory and per-step latency stay flat forever.

Two secondary ideas are worth noting. First, the clock itself is selectable via a learned frame-length embedding — the same weights run at 12.5, 8.3, or 6.25 decisions/second, trading granularity for compute. Second, it ships semantic VAD heads (8 classes, prediction horizons 0.5/1/2/3 s) meant to distinguish thinking pauses and stuttering from actual end-of-turn — the failure mode that plagues acoustic VAD in voice agents. The roadmap says the formal release will build frame-level semantic perception on this same grid and forward pass.

How it works

Frankenstein assembly, all components fine-tuned: a 32-layer causal audio tower initialized from Voxtral Realtime 4B (128 mel bins, sliding window 750), a randomly initialized projector, and a Qwen2.5-3B-Instruct decoder + LM head (36 layers, GQA 16/2). Decoding is DSM-style: the text stream trails the audio stream by N delay tokens; more delay = more right-context = better accuracy. Only certain (clock, delay) pairs are post-trained — e.g., at 80 ms the supported delays are 240/320/480/560 ms — though other multiples of the clock will run.

Evidence

Self-reported, greedy decode at 80 ms clock / 480 ms delay:

The Chinese gap is the headline — Voxtral's realtime model apparently collapses on Mandarin, and this checkpoint doesn't. On English it trails Voxtral by ~0.8–1.3 WER points, respectable given it's a bilingual model. No noise/accent/far-field evals are reported, and the "average" row mixes CER and WER across sets with different baseline coverage, which is a bit sloppy.

What's actually there, and caveats

Apache-2.0. Weights on HF (8.17 GB bf16 model.safetensors + separate semantic VAD heads), custom_code remote modeling code, a GitHub repo with a docker-compose stack running an adapted vLLM build with a WebSocket realtime endpoint and web demo client, plus a torch simulated-streaming script. Downloads (~20k) and likes (1.4k) suggest real early adoption.

Caveats: preview release; the semantic-perception features are shipped-but-forward-looking; the 24/7 no-drift claim is asserted, not benchmarked; the vLLM path is a fork, so it may lag upstream; only full merged weight dirs are supported (no adapters); the usage snippet duck-types the audio config, which hints the API is still rough. You'll want a GPU with ~16 GB+ for bf16.

Try it

Fastest path is the docker stack (serves the WS API and web demo):

cd docker
AUDIO8_MODEL_DIR=/path/to/checkpoint docker compose up -d
# demo at http://localhost:8080/, or drive it from a terminal:
python -m audio8_asr_infinite.examples.vllm_realtime_client \
  --ws-url ws://127.0.0.1:18191/v1/realtime \
  --audio sample.wav --language zh --target-delay-ms 480 --pace

Or pure-torch simulated streaming: python -m audio8_asr_infinite.examples.torch_streaming_decode --checkpoint ... --audio sample.wav --language en --transcription-delay-ms 480. Model card: https://huggingface.co/Edge0/Audio8-ASR-Infinite