AESSI: An Around-Ear Silent Speech Interface for Cross-Day Online Reuse without Test-Day Calibration

TL;DR: A dry-electrode array tucked behind each ear reads the electrophysiology of silent mouthing, and a frozen model — trained only on historical recordings — decodes which of 25 Mandarin sentences the user mouthed, weeks after the last training session, with zero test-day calibration: 92.2% offline on held-out later-day sessions, 98.0% in a prospective live test run at least 21 days after the last recording, with ~32 ms of CPU compute per prediction. The technical engine is a self-distillation scheme (MCRP) where a student predicts a frozen teacher's representations from masked spectrograms — and, strikingly, the student ends up ~15 points better than the teacher it imitated. The catch: a fixed 25-sentence menu, six participants in the offline evaluation, five in the online one.

The problem nobody wants to admit: day two

Almost every wearable neural or muscle interface has the same dirty secret. It works beautifully on the day you collect the data — and then you take the device off. Tomorrow, the electrodes land a millimeter differently, skin impedance has changed, you're tired, your articulation habits drift. The signal distribution for the exact same intent has moved, and the model you trained yesterday quietly falls apart.

The field's standard workarounds all concede the point. Inoue et al.'s around-ear silent-speech system — the closest prior work — had healthy users complete 640 labeled calibration trials on the test day before online use. SilentWear's neck-EMG system saw accuracy collapse on re-wear and patched it with fine-tuning on new-day data. Even studies that report "cross-session" results often use within-session train/test splits, where temporal autocorrelation and shared electrode contact inflate accuracy in ways that say nothing about tomorrow.

The paper frames this cleanly as domain generalization, not domain adaptation. Each electrode application is a domain, characterized by a latent session covariate z (placement, contact, physiological state); the observation x depends on both the sentence class y and z. Crucially, online use forbids the usual adaptation tricks: at test time you get one 3-second window and nothing else — no unlabeled test-day data to fit statistics, no fine-tuning. Everything, including preprocessing parameters and whitening statistics, is frozen from historical data.

The sensing choice matters for practicality: bilateral cEEGrid arrays — flexible printed electrode strips curling around each ear on relatively hair-free skin, 18 usable channels at 1 kHz. Around-ear electrodes sit near temporal, jaw, and facial muscles, so silent articulation produces detectable (largely EMG-flavored, mixed with EEG) signals, and the form factor could plausibly live inside a headphone. That's the appeal — and also why reapplication drift is severe: it's a skin-contact array you peel off and reapply.

The mechanism

Task and input. 24 native Mandarin speakers, 44 sessions, 11,000 trials. Each trial: the user silently mouths one of 25 six-character everyday sentences ("I am hungry and want to eat," "Please call the doctor") during a fixed 3 s window. Equal sentence length kills the cheapest cue. Each 3 s, 18-channel window becomes a log-STFT: 200 ms Hann windows, 50 ms hop, 100 bins from 5–500 Hz — an 18 \times 100 \times 61 tensor. Same representation at train and test, computable causally within the window.

Decoder. Four parallel branches view the same tensor differently: local time-frequency convolutions; per-frequency temporal trajectories (averaged over electrodes); per-electrode intensity trajectories (averaged over frequency); and a "Gram" branch that, per frame, computes the 18\times18 shrinkage-regularized Gram matrix of normalized channel spectra — capturing which electrodes have similar spectral shapes right now, a feature that's plausibly more robust to gain and placement changes than raw amplitudes. The branches fuse into a 192-dim sequence, pass through five dilated temporal residual blocks and self-attention, then multi-mode pooling and a 25-way classifier. Ablations show heavy redundancy: removing any one branch costs only 0.13–0.58 points. The architecture is not the story.

The story is the training pathway: MCRP. First, train a teacher with plain cross-entropy on the augmented data and freeze it. Then initialize an architecturally identical student from scratch, feed it inputs with six consecutive time frames and five consecutive frequency bins zeroed out, and train it to predict the teacher's intermediate representations of the complete input — at the fusion layer and residual blocks 3 and 5 — via a whitened cosine loss evaluated only at masked temporal positions:

where \mathcal{M} is the set of masked frames, \ell indexes the three alignment layers, \tilde H denotes representations normalized and whitened using teacher statistics estimated from training data, and \text{sg} is stop-gradient. After 10 epochs of this, the student is fine-tuned with cross-entropy. Only the student ships.

Why should this help with day-to-day drift? Electrode reapplication is essentially a natural masking/perturbation process: some channels degrade, some frequency bands shift, some local time-frequency cues that were reliable yesterday are gone today. A vanilla classifier is free to latch onto a handful of highly discriminative local features — which is exactly what breaks. MCRP structurally penalizes that strategy: any representation that depends on a small set of local cues cannot be predicted when those cues are masked. It's the BEATs/data2vec insight from self-supervised audio, but pointed at a different target — not learning from unlabeled data at scale, but manufacturing robustness to sensor drift from a small labeled corpus.

The evidence

The main evaluation is leave-one-day-out: for six participants with ≥3 sessions, the entire final session (a separate electrode application, different day) is held out; the model trains on the participant's earlier sessions plus all other users' sessions (personal examples weighted 8:1).

Two things jump out. First, the architecture alone (74.2%) already crushes the adapted baselines — though the authors are candid that baselines like InoueNet and SpeechNet were designed for settings with test-day calibration or non-causal filtering, so the 50.6-point headline gap over ATCNet partly reflects task mismatch. Second, and more interesting, is the training-pathway ablation:

Here's the aha: the teacher — the very network whose representations the student learns to predict — scores 77.2%. The student, trained to imitate it under masking and then fine-tuned, scores 92.2%. The student beats its teacher by 15 points on out-of-distribution data. Distillation here isn't compressing knowledge; the masked-prediction task itself is doing regularization work that no amount of information in the teacher explains. Meanwhile heavy input augmentation — the standard remedy for sensor drift — buys only 3 points. If this pattern replicates, it's a genuinely useful recipe for any skin-contact biosignal interface: EMG wristbands, ear-EEG, neckbands.

The online result is the paper's most credible contribution, because it's prospective and hard to fake. Five participants returned ≥21 days after their last recording. Models, preprocessing, and whitening were frozen beforehand. Targets were drawn live from RANDOM.ORG on an independent tablet; four camera angles plus screen capture recorded everything from electrode placement onward. Setup took ~5.5 minutes per person (glue on electrodes, check for dead channels — impedance not even rechecked), then straight to use. Result: 245/250 correct (98.0%; individual scores 46, 50, 50, 50, 49 out of 50). CPU replay of the raw trials: median 31.7 ms, 95th percentile 35.1 ms per prediction. The online accuracy exceeding the offline 92.2% likely reflects models trained on one extra session per user (the previously held-out one) plus small-n luck, but the direction is what matters: no degradation from three-plus weeks of shelf time.

Data economics. A separate experiment varies personal sessions H and population sessions G, always testing on the same later-day session. With H=1 and no population data: 13.3%. Same single personal session plus the full ~41-session population pool: 73.6%. With H=2: 35.1% → 88.6%. Even pure leave-one-participant-out (H=0, full pool) reaches 68.4%. Population data is worth 50–60 points when personal history is thin. A related transfer experiment shows population pretraining on only 10 shared sentences boosts supervised learning of 15 new personal sentences from 51.4% to 75.8% — the pretrained features generalize across sentence identity, not just across people. That's the seed of a real product path: ship a population model, ask each user for one or two short enrollment sessions, done.

What to be skeptical about

It's a 25-way menu, not speech decoding. Fixed candidate set, six characters per sentence, one holistic prediction per 3 s window. Nothing here tells you whether the representations support phoneme-level or open-vocabulary decoding; the sentence-transfer result is encouraging but the added classes were fully labeled during personal training. The comparison point that flatters AESSI — Inoue et al.'s 640-trial test-day calibration — was for a 64-word task, harder in vocabulary if easier in structure.

Small n where it counts. Six participants in the cross-day evaluation, five online, all healthy young adults, and the target participants are exactly the people motivated enough to return for 3+ sessions. The stated clinical motivation (post-stroke dysarthria) is entirely untested; dysarthric users may produce weaker or more variable articulation signals.

The signal is probably mostly EMG, and confounds aren't fully excluded. The authors acknowledge they don't separate EEG from facial/jaw EMG contributions, and that fixating on a cross doesn't rule out non-articulatory movement cues. For a communication device that's arguably fine — use whatever the electrodes pick up — but it matters for anyone hoping this transfers to imagined (non-mouthed) speech, where the EMG channel vanishes.

Baseline gaps overstate the method's novelty margin. The adapted baselines were built for other tasks and constraints; the fairest internal comparison is AESSI-without-MCRP vs. full AESSI, and that 18-point gap is the number I'd trust.

Why it matters anyway

Test-day calibration is arguably the adoption blocker for wearable electrophysiological interfaces — nobody will use a device that demands ten minutes of labeled drills every morning. This paper is one of the cleanest demonstrations I've seen that a frozen model on a reapplied skin-contact array can just work, weeks later, verified prospectively on video with externally randomized targets. And it locates the enabling ingredient not in the sensor or the architecture but in a cheap, general training recipe: masked-context self-distillation as a synthetic stand-in for sensor drift. That recipe costs 10 extra epochs and transfers, in principle, to any biosignal domain plagued by the same problem — which is nearly all of them.

Where to spend your reading time: Section 3.5 (the MCRP formulation, including the whitening details that make cross-layer alignment work) and Section 4.2 (the student-beats-teacher ablation) contain the transferable ideas. Section 3.11 is worth skimming for how to run a prospective online BCI evaluation that's actually convincing — independent random target generation, continuous multi-camera recording, frozen-before-visit models — a protocol more of the field should copy.