MiniMax-AI/MiniMax-Music3

TL;DR: MiniMax Music 3 is an open-weights song generation model that produces complete, structured songs — vocals, arrangement, five minutes, 32 kHz stereo — from lyrics plus a natural-language music description. The interesting part architecturally: an 8B "global" LM and 0.6B "local" LM predict RVQ tokens hierarchically, but the waveform is synthesized from the continuous hidden states of both LMs via a 2.4B flow-matching model, skipping the discrete tokenizer decoder entirely at inference. It's a real capability release in open music generation, though the README ships no evals and no stated license.

What it is

Text-to-song with two inputs: lyrics (with optional section tags like [Verse], [Chorus], [Bridge], [Instrumental]) and a music description covering style, vocals, instrumentation, and production. The pitch is long-range coherence — maintaining themes, vocal identity, and arrangement progression across a full intro-to-outro song structure, up to five minutes. Most open music models (MusicGen, YuE, ACE-Step, etc.) either cap out at shorter clips or degrade structurally over long generations, so if the coherence claim holds under listening, this is the headline capability.

How it works

Tokenization is 8-layer RVQ at the frame level, with an unusual split: the first codebook is a large 16,384-entry semantic codebook trained first, capturing musical structure; the remaining seven are 1,024-entry acoustic residual codebooks. The Global LLM (8B, initialized from Qwen3-8B with adapted embedding/output layers) autoregressively predicts the semantic codebook frame by frame — that's where long-range structure lives. The Local LLM (0.6B) fills in the seven acoustic codebooks within each frame. This global/local factorization is the familiar hierarchical trick (cf. MegaByte, Moshi's depth transformer), scaled up and specialized for music.

The genuinely notable choice is synthesis. Instead of decoding the discrete RVQ tokens back to audio, the final hidden states of both LMs are fused and fed to a 2.4B flow-matching model that predicts a Flow-VAE latent, decoded by a 123M decoder to 32 kHz stereo. The discrete tokens act as a training scaffold; at inference the quantization bottleneck is bypassed and the richer continuous representation carries vocal articulation, timbre, and temporal continuity to the vocoder. This "decode from hidden states, not tokens" pattern has been emerging in speech (and MiniMax says the Flow-VAE is adapted from their speech stack, retrained for music's dynamic range) — seeing it in an open long-form music model is a useful data point that it scales.

Discrete tokens structure the autoregressive modeling; the audio path runs entirely on continuous representations, avoiding RVQ quantization loss at synthesis time.

What's actually there

Weights on Hugging Face (MiniMaxAI/MiniMax-Music3), a serving path via SGLang-Omni, one example WAV in the repo, a hosted demo page, and a music-caption-rewriter agent skill that expands a short prompt into a structured caption (Global Metadata / Vocal Details / Arrangement) — fully offline, using a bundled style library. What's not there: no benchmark numbers, no listening-test results, no comparison to Suno/Udio/YuE, no training data description, no paper, and — importantly — no license stated in the README. Verify the model card before any commercial use.

Practicalities and caveats

Inference needs two CUDA GPUs: GPU 0 runs the Qwen3-based AR generation, GPU 1 runs flow matching and waveform decoding. Given ~11B total parameters plus 5-minute KV caches, expect serious VRAM. Non-streaming only; prompts capped at 5,000 tokens, generation at 9,000 acoustic frames. The README is honest that section tags and descriptions are "generative control rather than strict symbolic guarantees" — BPM, key, and structure requests may drift. All quality claims are self-reported prose; the only evidence you can check is the demo page and the bundled sample, so listen before believing the coherence story.

Try it

hf download MiniMaxAI/MiniMax-Music3 --local-dir /path/to/minimax_ttm
# from the SGLang-Omni repo root:
sgl-omni serve --model-path MiniMaxAI/MiniMax-Music3 --port 8000

curl http://127.0.0.1:8000/v1/audio/speech \
 -H 'Content-Type: application/json' \
 -d '{"model": "minimax_ttm",
      "input": "[Verse]\nMorning light filtering through the pine\n[Chorus]\nSoftly the world begins to breathe",
      "instructions": "Warm acoustic pop, intimate female vocals, fingerpicked guitar, building to a wide final chorus.",
      "response_format": "wav", "seed": 7, "max_new_tokens": 9000}' \
 --output minimax_music3.wav

Zero-setup option: the hosted demo linked from the repo.