Paper Feed

Issue 23 · Project 03 GitHub BCI ✓ read

facebookresearch/brain2qwerty

Non-invasive decoding of typed sentences from MEG and EEG brain recordings using a convolutional encoder, transformer, and character-level language model.

GitHub ↗ ·★ 917 ·131 forks·Python ·created 2026-06-04 ·3 min read

TL;DR: Brain2Qwerty is Meta AI's open-sourced pipeline for decoding what someone is typing from non-invasive brain recordings — MEG and EEG — by mapping neural signals through a convolutional encoder and transformer into character sequences, cleaned up by a character-level language model. It's the code behind a Nature Neuroscience 2026 paper plus a follow-up preprint, released with a Spanish MEG/EEG dataset. This matters because sentence-level brain-to-text has so far been the domain of invasive implants; showing it works at all without surgery is a real capability marker for BCI, even with heavy caveats.

What it is

The task setup: participants type sentences on a QWERTY keyboard while their brain activity is recorded with MEG (magnetoencephalography) or EEG. The model decodes the character sequence being typed directly from the neural signals — not from muscle activity or keystroke timing. So it's decoding the motor/linguistic intent chain around typing, at the sentence level, from outside the skull.

That's the interesting framing: rather than decoding attempted speech (the invasive-implant route from the Stanford/UCSF lineage) or imagined handwriting, this targets a motor behavior almost everyone already has — typing — and asks how far non-invasive sensors can get. Two generations are represented: v1 (the Nature Neuroscience paper, "Non-invasive decoding of typed sentences from human brain activity") and v2 (a 2026 preprint, "Accurate Decoding of Natural Sentences from Non-Invasive Brain Recordings"), each with its own code directory.

How it works

The README is sparse on architecture detail, but the stated design is a three-stage stack:

MEG / EEG multichannel time series Conv encoder local spatio- temporal features Transformer sentence-level context Char-level LM → text Neural priors from the encoder + linguistic priors from the LM jointly constrain the decoded sentence
The stack: a conv net extracts features from raw sensor time series, a transformer integrates over the sentence, and a character-level LM imposes linguistic structure on the output. The LM is doing real work here — raw per-character neural evidence is noisy, and sentence-level decoding leans on language priors.

The repo also references two infrastructure components, NeuralSet and NeuralTrain — presumably data-loading and training frameworks for neural recordings — but the README doesn't describe them.

What's actually released

  • Code: brain2qwerty_v1/ and brain2qwerty_v2/, Python, under CC BY-NC 4.0 (non-commercial — notable that they used a data-style license for code rather than MIT/Apache).
  • Data (v1): a Spanish-language MEG/EEG dataset on Hugging Face (bcbl190626/SpanishBCBL), collected by and belonging to BCBL, the Basque Center on Cognition, Brain and Language.
  • Data (v2): embargoed until paper acceptance — so the newer, presumably better system can't yet be fully reproduced.
  • No numbers in the README. No CER/WER tables, no model cards, no pretrained weights mentioned. For the actual accuracy figures you have to go to the papers. From what's publicly known about this line of work: MEG substantially outperforms EEG (the magnetic signal is far cleaner), and best-participant results are usable-looking while averages are not — read the per-modality, per-participant error rates in the paper before forming an opinion.

Caveats

Be clear-eyed about what "non-invasive" buys you here. MEG requires a magnetically shielded room and a machine the size of a dental chair; this is not a wearable, and the practical BCI story runs through EEG, which is the much weaker modality in this comparison. Also: participants are physically typing, so this decodes brain activity around overt motor production — the step to imagined typing (the case that matters for paralyzed users) is not demonstrated here. The Spanish-only dataset and the v2 embargo further limit what you can build on today.

Still, as an open, reproducible baseline for non-invasive sentence decoding — with real data released — it's the reference point this subfield needed.

Trying it

No install instructions or quickstart in the README. The realistic path:

git clone https://github.com/facebookresearch/brain2qwerty
cd brain2qwerty/brain2qwerty_v1
# dataset (v1):
# https://huggingface.co/datasets/bcbl190626/SpanishBCBL

Expect to read source to reconstruct the training pipeline; check the v1 directory for configs and the two papers for architecture and evaluation details.