espnet/yodas3

TL;DR: YODAS v3 is ESPnet's third crawl of CC-BY-licensed web audio: 1.1M+ hours across 102 languages, ~75% with timestamped transcripts, ~60% of non-English data with English translations, plus unusually careful quality metadata (effective bandwidth, true channel count). It's arguably the largest openly-licensed speech corpus in existence, and — unlike YouTube-derived datasets distributed as URL lists — the audio itself is hosted on HF. If you train speech or audio models, this will be in your data mix within a year.

What it is

The YODAS series crawls videos whose uploaders chose a CC-BY-3.0 license, which is what lets ESPnet redistribute the raw audio rather than links. v3 is a fresh crawl guaranteed disjoint from v1/v2, so you can stack them. The headline claim from the Interspeech 2026 paper title — "high-bandwidth, stereophonic, multilingual" — is the interesting part: prior web-speech corpora are treated as 16 kHz mono ASR fodder, whereas v3 is explicitly positioned for TTS and general audio representation learning by measuring what the audio actually contains.

Concretely, they ran an FFT on a 10s excerpt of every file to estimate effective sample rate (catching upsampled junk) and compared channels by RMS difference to catch dual-mono stereo (~25% of files). Over 92% of the data has ≥32 kHz effective bandwidth and 67% reaches 44 kHz; over 70% has genuinely distinct stereo channels. That's a per-item quality filter no other corpus at this scale ships.

How it's packaged

Layout is WebDataset tars (~8 GB each, 6,685 shards total — so roughly 50+ TB of Opus/WebM) paired 1:1 with parquet metadata files: data/{lang}/audio/0042.tar contains exactly the rows in data/{lang}/metadata/0042.parquet, joined on a 32-hex id. That makes streaming pipelines trivial — you never need the whole thing on disk, and you can pull metadata alone to plan a subset.

Transcripts are JSON segment lists with millisecond sentence timestamps; ~95% (the ASR-caption ones) also carry word-level offsets. English translations are segment-level only. Language IDs come from two sources — caption-track language (used for the directory layout, more accurate) and uploader locale (what the paper reported) — and both are in the metadata, which is an honest touch.

What to watch

• Labels are web captions. Most transcripts (and all word timestamps) come from YouTube's auto-ASR, so this is closer to pseudo-labels than gold truth. Fine for pretraining and TTS data mining; spot-check before treating it as eval data or fine-tuning targets in your language.
• LID is file-level and admittedly imperfect at sentence level; the two ID sources disagree enough that the team changed which one they trust after the paper.
• License diligence. CC-BY-3.0 flows from uploaders' license selection; that's per-item in principle, but uploaders sometimes mislicense content they don't own. Standard web-crawl caveat, and CC-BY also requires attribution, which is awkward at 1.1M items.
• Scale of access: full dataset is ~50 TB; the per-language --include filters and metadata-only download are the sane entry points. 34 languages exceed 1,000 hours, so mid-resource languages (Hindi, Tamil, Bengali in their examples) are a real draw.
• The parallel translation pairs (~1.07M videos) are an underrated asset for speech translation and any-to-any models — that's a lot of aligned speech→English text.

Try it

Metadata first, then one language:

# plan your subset: all metadata, no audio
hf download espnet/yodas3 --repo-type dataset \
  --include "data/*/metadata/*" --local-dir yodas3

# then pull one language's audio
hf download espnet/yodas3 --repo-type dataset \
  --include "data/hi/*" --local-dir yodas3

Or stream without downloading, joining tars to transcripts:

from datasets import load_dataset

meta = {r["id"]: r for r in load_dataset("espnet/yodas3", "hi_metadata", split="train")}
audio = load_dataset("webdataset",
    data_files="hf://datasets/espnet/yodas3/data/hi/audio/*.tar",
    split="train", streaming=True)
for s in audio:
    m = meta[s["__key__"]]   # transcript, translation_en, bandwidth, channels...
    break

Dataset card: https://huggingface.co/datasets/espnet/yodas3