BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

TL;DR: A 150M-parameter recurrent model solves ARC puzzles by ingesting the demonstration pairs into a persistent recurrent memory, then iterating on a continuous latent state until it can decode an answer — no chain-of-thought tokens, no test-time gradient steps, no task-specific fine-tuning. It scores 29.5% pass@2 on the public ARC-AGI-1 eval set at a computed $0.0007 per task, roughly 57× cheaper than the cheapest frontier LLM at comparable accuracy, and ~2000× cheaper than HRM/TRM-style recursive solvers that need per-task optimization. The genuinely interesting part isn't the score — it's the demonstration that in-context learning and latent (non-verbal) reasoning can live in one system, plus an unusually careful battery of controlled probes showing exactly what such a system can and cannot infer from examples. The main caveats: the architecture's internals are proprietary, and all evidence comes from one narrow task family.

Two things that never got combined

There are two well-established ways to get a model to do something new at inference time, and they've lived in separate research lineages.

The first is in-context learning with chain-of-thought: LLMs read demonstrations, then think out loud. This is flexible but expensive in a specific, structural way — every intermediate state of the computation must be squeezed through a discrete vocabulary, emitted one token at a time, and re-read before computation can continue. The reasoning trace is the working memory, and it's serialized through a lossy, low-bandwidth channel.

The second is latent recurrent reasoning: models like Coconut, recurrent-depth transformers, and the HRM/TRM family iterate on a continuous hidden state instead of emitting tokens. This keeps partial hypotheses in high-dimensional superposition — theory work (Zhu et al., 2025, cited in the paper) shows a single continuous state can carry multiple search frontiers in parallel. But on ARC specifically, these recursive solvers have been transductive: HRM and TRM train on augmented demonstration pairs from the evaluation tasks themselves, with a learned identity embedding per task. A genuinely unseen task requires backward passes before the model can even attempt it. That's why ARC Prize reports their costs at 1.48–1.76 per task — you're paying for optimization at test time, and you're not really doing in-context learning at all.

BDH-CQ (built on the authors' earlier "Dragon Hatchling" architecture — brain-inspired, high-dimensional sparse positive activations, recurrent associative state) targets exactly the missing conjunction: acquire an unseen transformation purely from demonstrations in the forward pass, then solve it by iterating in latent space. No task IDs, no eval-task pairs in training, no parameter updates at inference.

The mechanism: a slow memory and a fast workspace

The system separates two kinds of state with different jobs. Demonstrations D_t (input–output grid pairs) are processed sequentially, each one updating a recurrent memory:

where \theta is frozen. S plays the role that a KV cache plays in a transformer — it holds context-dependent associations — but as a fixed-size recurrent state rather than a growing cache. (The paper notes linear attention is the simplest special case, where the update degenerates to S_t = S_{t-1} + U_\theta(D_t).) This is fast-weights territory: the demonstrations effectively write a temporary program into S.

Then, with all K demonstrations absorbed, the query x^\star is encoded into a latent workspace and iterated R times against that frozen memory before decoding:

The analogy that fits: S is what you learn from watching someone demonstrate a trick — a temporary rule, held without changing who you are (\theta fixed). H is the scratch of actually performing it: an evolving mental image you refine until it settles, and only then do you speak.

An honest structural caveat up front: the paper reports the interface (these equations) but states that "dimensions, exact update rules, and implementation details remain proprietary," as does the full training recipe. This is a system report with controlled behavioral science on top, not a reproducible architecture paper. Training used a curated mixture of ARC-style data (ARC-AGI-1 training set, RE-ARC, ConceptARC, ARC-Heavy, ARC-GEN100K, plus private data and augmentations) — so the in-context learning ability is meta-learned over a distribution deliberately shaped like the evaluation domain.

The headline number, in context

On the 400-task public ARC-AGI-1 evaluation set, the 150M model reaches 29.5% pass@2 in ~0.85 H200 GPU-seconds per task, which at 3/H200-hour computes to **0.00070 per task**. Against the ARC Prize leaderboard, no plotted system reaches this accuracy at equal or lower cost. GPT 5.6 Luna (Low) scores 34.2% at 0.040/task — 57× more expensive at leaderboard prices, 11× after OpenAI's subsequent price cut. HRM and TRM, the recursive solvers closest in spirit, cost1.48 and $1.76 per task because they optimize per task. An independent black-box audit by co-authors at Bielik and NYU reproduced the 29.5% under a documented protocol without weight access, and repeated identical requests were byte-identical — a nice touch of determinism that most LLM evaluations lack.

Is 29.5% good in absolute terms? No — the average human solves most ARC-AGI-1 tasks, and frontier LLM systems with heavy test-time compute exceed 80%. The claim is about the Pareto frontier: this operating point is three to four orders of magnitude cheaper than anything comparable, and the paper argues the gap is headroom — you can scale this architecture a lot before losing cost-competitiveness. Section 9.2 asserts (without shown data) that pretraining scaling laws hold from 1B to 600B parameters for the architecture family.

The paper also shows a clean latent-compute dial: the model was trained across multiple reasoning-effort levels, and at inference you choose how much latent iteration to spend. More iterations, better answers — test-time compute scaling without a single generated token.

What the probes actually reveal

The most valuable part of the paper for an ML researcher is Section 6: after freezing the model, the authors built controlled ARC-like task generators that vary one factor at a time. This is behavioral psychophysics for a reasoning model, and it produces a much sharper picture than a benchmark score.

Contextual binding is strong and dense. Tasks defining a fresh color permutation through demonstrations — pure fast-weight writing — are solved 96/96 at rank one, even with eight simultaneous bindings. The memory mechanism works.

Some operators extrapolate; others hit walls with distinct signatures. Boundary propagation stays perfect (48/48) as distance extends well beyond the demonstrated range; motif copying likewise as target sites multiply. But ordering (sort bars by height) collapses hard past six objects, and nested containment has a cliff at depth five:

The diagnosis of why these two fail differs, and the follow-up experiment is the paper's best move. At ordering length eight, only 3/24 outputs even have the correct dimensions — the model can't construct the output at all. At nesting depth five, all 36 outputs have correct dimensions and >99.9% cell accuracy; errors are typically a single wrong containment decision. Then they rerun byte-identical test inputs with one demonstration added at the target complexity: nesting jumps from 19/24 to a perfect 24/24, and ordering recovers from 0/24 to 13/24. So the nesting cliff is almost entirely a failure to extrapolate beyond demonstrated relational depth — the machinery can execute depth five if shown depth five — while long ordering has a genuine execution-capacity bottleneck that support only partially fixes. That's a clean dissociation between "didn't infer the rule generalizes this far" and "can't run the rule."

Composition is representation-dependent, not uniformly hard. Each atomic operation on a 3×3 motif — relocation, reflection, rotation — is solved 72/72. Composed with relocation: rotation stays perfect (72/72), reflection drops to 47/72, and a demonstration-defined color swap composes at exactly 0/72 (and is only reliably acquired atomically in one motif family, 26/72 pooled).

Conditional structure is the sharpest weakness. In the appendix ladders: when a corner marker must select between two demonstrated rules, accuracy drops from 100% (marker varies but both values invoke the same rule) to 56.7% — the cost attaches specifically to using a cue for rule selection. Worse, when a transformation parameter (a shift magnitude specified by marker count) takes a value never shown in demonstrations, accuracy is 0/120 — including values interpolated between demonstrated ones. The system binds demonstrated (cue → behavior) pairs; it does not abstract the parametric rule connecting them. Panel-union tasks fall off a cliff from two panels (26/40) to three (1/40), and even two touching panels score 3/40 versus 26/40 for well-separated ones — smelling like a segmentation limit.

Consistency is a real gap. On ConceptARC (59.4% strict-task, 77.9% test-pair pass@2), 52 of 160 tasks have one or two of three test inputs correct but not all three. Under a rule-induction account, a correctly induced rule should transfer to every test input of its task; this says the model often produces correct outputs without holding a stable rule. That's the honest reading of what "reasoning" means here — closer to a strong pattern-completion prior conditioned by context than a symbolic rule engine.

What changes if this holds — and what to doubt

If the approach scales the way the authors project, the interesting shift is economic and architectural: reasoning as latent iteration against fast-weight task memory rather than token generation. A fixed-size recurrent state instead of a growing KV cache, deterministic outputs, a compute dial with no vocabulary in the loop, and cost per task measured in hundredths of a cent — that's a genuinely different operating regime, and it revives fast-weights/linear-attention ideas (Ba et al. 2016, Katharopoulos et al. 2020) in a setting where they demonstrably do the in-context-learning job. The paper also gestures, plausibly, at hybrids: BDH layers support language modeling, so a future system could reason in latent space and decode language only when communication or tool use demands it — echoing the Fedorenko-line evidence that human language and reasoning are partly separable systems.

Reasons for skepticism, in rough order of weight:

1. One domain, and the training distribution was built for it. Every result is ARC-style grids, and the training mixture is ARC-shaped by construction. Whether this in-context latent reasoning transfers to language, math, or anything with less regular structure is entirely untested — the paper's own outlook section concedes this. The conditional-selection and unseen-parameter failures suggest the current system would struggle badly with tasks requiring genuine parametric abstraction.
2. Proprietary internals. You cannot rebuild this from the paper. The equations are an interface, not a method; the training recipe, dimensions, and update rules are withheld. The "evaluated system includes input transformations, candidate construction, ranking, and the inference pipeline" — an unknown amount of the score may live in that pipeline rather than the model.
3. Cost comparisons are apples-to-oranges by nature. BDH-CQ's cost is computed from measured GPU-seconds at $3/hour; competitor costs are API prices or hardware estimates, which mix margins into one axis. The frontier-breaking claim is robust to this (three-plus orders of magnitude of slack), but the exact multipliers are soft.
4. The generated-task analyses carry authoring confounds the authors themselves flag: hand-written gravity tasks score 80–100% versus 2.9% for GPT-generated ones — mechanic-level solve rates characterize the task generator as much as the model.
5. Some claims are unaudited: the 1B–600B scaling-law statement in Section 9.2 comes with no data.

To the paper's credit, most of these caveats appear in its own text — the behavioral sections are unusually candid about confounds, preregistered replications, and what each experiment does not isolate.

Where to spend your time

Read Section 6 (the controlled probes) and Table 3 — the short-vs-supported context experiment is the single most informative result, cleanly separating extrapolation failure from execution failure. Then Appendix A.3 for the conditional-selection and unseen-parameter ladders, which are the clearest map of where demonstration-conditioned latent reasoning currently ends. Section 3 takes five minutes; the headline result in Section 5 you already know.