Accurate Decoding of Natural Sentences from Non-Invasive Brain Recordings

TL;DR: A Meta/PSL team decoded naturally typed sentences from non-invasive MEG at 39% word error rate on average — 22% for the best subject, with half of that subject's sentences decoded with at most one wrong word. Two years ago the non-invasive state of the art couldn't reliably produce a single coherent sentence. The recipe: 10× more data per subject than prior work, a CTC-based asynchronous encoder that needs no keystroke timing, a clever trick for turning continuous brain-signal embeddings into word-level tokens an LLM can read, and — genuinely novel for this field — AI coding agents that autonomously improved the training pipeline. The headline claim to interrogate is the log-linear data-scaling curve, which shows no plateau at 90 hours and is the whole argument for "maybe we don't need implants."

The problem: no timestamps, no sentence

Invasive brain-to-text is remarkably good now. Utah-array typists reach under 2% WER; handwriting decoders reach under 6% character error rate. But every one of those systems requires opening someone's skull, and electrode arrays degrade under neuroinflammation. The non-invasive alternatives have been stuck: EEG is too noisy, fMRI too slow (Tang et al.'s celebrated 2023 fMRI decoder produced paraphrases at 92–94% WER on exact words), and MEG — which has good temporal resolution and decent spatial coverage — was until recently only useful for classification-style tasks.

The immediate predecessor, Brain2Qwerty v1 (Lévy et al. 2025), decoded typed characters from MEG at 32% CER. But it had a structural flaw: it was synchronous. It classified fixed windows time-locked to each keystroke, meaning the decoder needed to know exactly when every key was pressed. That's fine in a lab where a keyboard logs the presses; it's useless for a paralyzed patient, who is the entire point. And a stream of 68%-correct characters with no language prior mostly produces gibberish.

So the real problem is: decode from a continuous, unsegmented stretch of brain signal, and produce an actual sentence, not a character soup.

Insight one: scale unlocks asynchronous decoding

The paper's first move is a data collection effort that looks small by ML standards but is large by MEG standards: nine healthy right-handed touch-typists, 10 hours each in a 306-channel cryogenic MEG scanner, each hearing a sentence through headphones, waiting through a delay, then typing it from memory on an MR-compatible keyboard — ~22,000 sentence productions total, drawn from 2,724 unique Llama-generated sentences. That's 10× the per-subject data of v1, and crucially, almost every sentence is unique rather than repeated.

For asynchronous decoding they use Connectionist Temporal Classification (CTC) — the same alignment-free objective that speech recognition used before attention took over. CTC lets a model emit a character sequence from a continuous signal without ever being told where each character lives in time; it marginalizes over all monotonic alignments.

The scaling result is the paper's empirical backbone. On the old 1-hour-per-subject dataset, the asynchronous encoder is much worse than the synchronous one (59% vs 39% CER) — CTC is a harder learning problem. On the new dataset, the gap collapses to 2 points (25% vs 23%). And when they retrain on progressively larger fractions of the data, CER falls log-linearly with recording hours: Pearson r = -0.99, slope of -0.39 CER per decade of data, no saturation at the 90-hour ceiling.

A subtler and arguably more interesting finding: diversity is an independent axis of data quality. Holding total sentence count fixed, training on 256 unique sentences beats training on 128 sentences repeated twice (45% vs 65% CER, p < 0.001) — even for a pure encoder with no language model attached. Repeated sentences apparently invite the encoder to memorize sequence-level shortcuts rather than learn keystroke representations. If you're designing a BCI data-collection protocol, this single panel (Figure 1G) changes what you should collect.

Insight two: teach an LLM to read neuro-tokens

Here's the architectural aha. Character-level CTC output ignores everything an LLM knows about language. The obvious fix — pipe CTC text through a language model as a spell-checker — is exactly what v1 did with an N-gram, and what fails when the character stream is too corrupted to rescue. The deeper fix is to give the LLM direct access to the neural signal. But LLMs eat discrete word-level tokens, and an MEG encoder emits a continuous embedding stream at 25 frames per second. How do you bridge that?

The paper's answer is almost embarrassingly simple: use the spacebar as a tokenizer. Spaces are 19% of typed characters and are robustly predicted by the CTC head. Wherever the greedy CTC path emits a space, cut the continuous MEG embedding stream. Each chunk between spaces is a candidate word: mean-pool its frames, push through a small MLP into the LLM's word-embedding space. The word count of the resulting token sequence lands within ±1 of the ground truth for 86% of sentences.

Those neuro-tokens get aligned to real word embeddings with a contrastive SigLIP loss, using dynamic time warping to handle the occasional off-by-one segmentation. Then a Qwen3-4B model, fine-tuned with LoRA adapters, receives a spartan prompt — CTC: <noisy text> MEG: <neuro-tokens> Output: — and autoregressively generates the sentence.

Does the LLM actually read the neuro-tokens, or is it just a fancy spell-checker? The ablation is clean: remove the MEG tokens and WER degrades from 0.39 to 0.49 (p \approx 0.004 on all metrics). The retrieval analysis is even more convincing — cosine-nearest-neighbor lookup from a neuro-token into Qwen's vocabulary recovers the correct word at rank 1 for 8 of 9 words in a typical sentence. This is a fine-tuned LLM that has genuinely learned to condition on brain activity, from only ~2,700 unique sentences.

The numbers, and an honest trade-off

The full model wins decisively where it matters for communication — words and meaning — but is worse at the character level (0.31 vs 0.26 CER for the N-gram). This is the classic LLM failure mode transplanted into neuroscience: when the encoder signal is too corrupted, the model doesn't produce garbled-but-close text; it produces a fluent, grammatical, entirely wrong sentence. The paper's own example from the worst subject: target "cars are not allowed on this road", decoded as "had she not fallen down the stairs." For a dialogue application that's arguably fine; for typing a password it's catastrophic. The authors flag this trade-off explicitly, which is to their credit.

Per-subject headline numbers: the best subject decodes 28% of test sentences with zero word errors and 47% within one edit; the median subject decodes 15% perfectly; the worst subject only 4%. That spread (encoder CER ranges 17–41% across subjects) is the field's persistent headache.

Two engineering ideas likely to travel

Model soup over per-subject LoRA adapters. How do you fine-tune one LLM for nine brains? Joint training on pooled data overfits once LoRA rank exceeds ~2 (the sentence pool is tiny). Their alternative: treat each subject as a separate task, train nine independent LoRA adapters, then uniformly average the adapter weights (à la Model Soups). The soup never enters the overfitting regime, keeps improving with rank up to 128, and — unlike joint training, which plateaus at WER ≈ 0.41 — keeps improving with backbone scale (0.6B → 1.7B → 4B, each step significant) and broader adapter targets. This "adapt per task, merge in weight space" recipe should generalize to any multi-subject neural decoding setup.

AI agents as pipeline optimizers. The most unusual section: they gave three Claude-Opus-powered Cursor agents filesystem and SLURM access, a validation-WER objective, and 10 rounds × 50 jobs each, starting from a stripped 4-hyperparameter config. An Optuna TPE baseline with identical compute plateaued at 0.41 validation WER; all three agents reached 0.36–0.38 by writing new code — independently rediscovering label smoothing, modality dropout (randomly hiding the CTC text so the LLM must rely on neuro-tokens), beam search, and minimal prompts. Critically, the agents' configs generalized to the other eight subjects (test WER 0.42–0.45, all p < 10^{-6} vs baseline) while Optuna's single-subject gain evaporated entirely (p = 0.88). Equally important is the negative result: given the open-ended task "start from the v1 codebase, match v2," the agents failed completely — crashed jobs, idle loops. Agents excel at constrained local optimization within a working pipeline; they cannot yet do the research.

How seriously to take the "path to non-invasive BCI" claim

The scaling argument is the paper's beating heart, and it's a five-point fit within one dataset spanning roughly one order of magnitude. Log-linear at slope -0.39 CER/decade sounds great until you compute what closing the gap to invasive systems (<2% WER) would require — several more decades of data, i.e., thousands of hours per deployment, assuming no plateau. The authors say "partially bridged," which is fair; the extrapolation itself is unvalidated.

Bigger caveats, in rough order of severity:

These are healthy people actually pressing keys. The MEG signal is dominated by motor cortex activity during real finger movements (their own source reconstruction shows bilateral M1/SMA). A locked-in patient produces attempted movement with no keypresses — no ground-truth labels for training, and plausibly weaker or reorganized motor signals. Invasive BCIs crossed this bridge; whether MEG's coarser signal survives the transition is completely untested here.

The hardware is a cryogenic 306-sensor system in a shielded room. The sensor ablation offers real hope, though: randomly dropping half the sensors costs only ~5.7 points of WER, well within the channel counts of wearable optically-pumped magnetometer helmets.

Subject-specific training is heavy but not absolute. Each subject contributed 10 hours, and per-subject adapters matter. But the appendix's leave-one-out experiment (Table 1) is encouraging: pretraining on eight subjects and fine-tuning on the ninth closes most of the gap to joint training (best subject: 32.8% WER vs 22.6% joint vs 38.3% from-scratch). New users won't need to start from zero.

Not real-time. The architecture is non-causal — it consumes the whole sentence before decoding. Users would see nothing until sentence end. Causal Conformers work for EMG handwriting, so this seems tractable, but it's future work.

Test distribution is narrow. Train/test splits are cleanly separated by sentence text (no leakage), but everything comes from the same pool of simple, contraction-free, Llama-generated sentences. The LLM's priors are well-matched to this distribution; open-vocabulary spontaneous language would be harder.

What changes if this holds

The framing shift is the important part: non-invasive brain-to-text stops being a curiosity that produces vague paraphrases and becomes a system whose errors look like a bad speech recognizer's — mostly single-word substitutions in otherwise correct sentences. The gap to implants goes from qualitative to quantitative, with a measured (if extrapolated) exchange rate between data and accuracy. Combined with the OPM-robustness result, there's now a concrete engineering roadmap — wearable sensors, cross-subject pretraining, causal decoding, attempted-movement transfer — rather than a physics argument about why MEG can't work.

For methods people, the two reusable ideas are the CTC-space tokenizer (a general recipe for handing continuous CTC-decoded streams to an LLM — plausibly useful for EMG, ECoG, even non-neural sequence modalities) and per-subject-LoRA-then-soup for multi-user model personalization.

Where to spend your reading time: Section 2.3 ("From MEG to LLM: align by word, adapt by subject") plus Figure 4 contains the transferable methodology; the appendix tables (multi-subject transfer, sensor ablation) contain the evidence that matters most for clinical plausibility. The Auto Research section (2.4) is a fun, well-controlled case study worth ten minutes regardless of your interest in BCIs.