Twin: Playing an Unknown Game with a Test-Time Digital Twin

TL;DR: Twin plays the ARC-AGI-3 benchmark — interactive grid games with hidden rules and hidden win conditions — by having a coding agent write a Python simulator of the game at test time, then enforcing a hard rule: no action gets submitted to the real game until that simulator exactly replays every transition observed so far. Every misprediction becomes an automatic counterexample for repair; every plan is verified inside the simulator before it costs anything. This takes the same base model from 7.8% (direct play) and 61.1% (off-the-shelf coding harness) to 93.3%, clearing 179 of 183 levels and beating first-time humans on action efficiency in 158 of them. The surprising secondary finding: recovering the dynamics of an unknown world turns out to be the easy part. Inferring what counts as winning is where all the remaining difficulty lives.

The problem: a game with no manual and no reward until you win

ARC-AGI-3 is the interactive successor to the ARC reasoning benchmarks. Each game is a 64×64 grid of colored cells. You get four to six actions (buttons plus a click at a coordinate), and after each action you see the next frame. That's it. Nothing tells you what the buttons do, what the objects are, or what ends a level. Humans clear these games readily; frontier models played directly mostly flail — the base model here (GPT-5.6 Sol) scores 7.8 out of 100 on its own.

The scoring is important: it's action efficiency against first-time human players. Reasoning, code execution, and simulated rollouts are free; only actions submitted to the real game count. So the benchmark implicitly rewards exactly one thing — knowing what an action will do before you take it. Which is to say: having a world model.

The obvious framing is reinforcement learning with sparse reward, and the obvious approaches are either learned latent dynamics models (Dreamer, MuZero — needing far more data than a few hundred interactions) or LLM agents that keep the model in the decision loop (ReAct-style), reacting to each frame with fresh reasoning. The 61.1% score of the raw Codex harness shows how far reactive reasoning gets you: it clears half the games and burns actions on trial and error.

The key idea: the world model is a program, and consistency is a hard gate

Twin's move is to treat the unknown game as a program synthesis target. The agent maintains a Python file — the "twin" — with a fixed contract:

step(grid, action) -> grid      goal_reached(grid) -> bool

That is, an executable hypothesis \hat{T} for the transition function and \hat{R} for the (never-stated) win predicate. It starts as an identity stub. Every real transition (s, a, s') is appended permanently to a log \mathcal{D}, and the harness enforces:

No scored action is allowed while this check fails. This is the single design decision that separates Twin from the five concurrent systems attacking the same benchmark. Others (EWM, OPINE-World, Schema) also synthesize executable models and test them against experience — but they ask the agent to validate via prompting. Twin's harness mechanically blocks action until the twin replays the entire history cell-for-cell. On 64×64 discrete grids, exact replay is free and decidable, so this is a cheap, airtight oracle.

The result is a beautiful closed loop, essentially CEGIS (counterexample-guided inductive synthesis) wrapped around an environment instead of a spec:

Execution is also instrumented. For a plan [a_1, \dots, a_k], the harness commits the predicted next frame (hash-stamped, so it can't be revised) before each submission and halts at the first index where reality disagrees. Every committed action therefore yields exactly one of two things: verified progress, or a localized counterexample. Nothing is wasted. In practice, 92.9% of scored actions execute plans already tested inside the twin; the remaining 7.1% are deliberate probes to reduce model uncertainty.

The asymmetric objective: dynamics are supervised, goals are not

Here's the framing I found genuinely clarifying. Every single action provides a labeled training example for \hat{T} — you took action a in state s and saw s'. But \hat{R} gets a positive label only when a level completes, which happens once per level. Over the full run, that's 11,557 dynamics labels versus 179 goal labels: a 65-to-1 supervision gap.

So Twin splits the objective asymmetrically. Consistency of \hat{T} with the log is a hard precondition (verifiable, cheap). Goal reachability is treated as optimism under uncertainty: hypothesize a goal, plan the cheapest route to test it, and let reality vote. When the twin validates but breadth-first search finds no goal state, a "goal discovery" branch searches the twin with a bigger budget (depth 14, 30,000 nodes vs. the planner's 8 and 20,000) and ranks reachable states by five generic visual-progress signals — a color vanishes, a new color appears, a compact region bursts with change, the scene shifts globally, or the search hits a new frontier. The coding agent then writes a candidate goal_reached predicate for the top state.

Two free filters apply before any scored action is spent. First, since ARC-AGI-3 replaces the winning frame when reporting completion, every frame in the log is a certified negative — so any candidate predicate that fires on a logged frame is rejected immediately. Second, any state previously reached without triggering completion is permanently blacklisted. Surviving candidates get tested by executing the cheapest route: a level boundary confirms the hypothesis; reaching the predicted state with no boundary refutes it; a transition mismatch en route says nothing about the goal and just triggers dynamics repair.

The analogy that fits: the dynamics half is unit-tested code with an ever-growing regression suite reality writes for you; the goal half is hypothesis-driven science, where each experiment costs real money (scored actions) and you design the cheapest decisive one.

The evidence

The headline numbers, all on the 25 public games (183 levels), all systems scored against the same first-time-human baselines:

Twin clears 23 of 25 games and hits the 100-point ceiling on 18. The base-model-controlled comparison is the clean one: same model, same file bridge, same sandbox, with the entire validate–explore–plan machinery deleted, drops from 93.3 to 61.1 and from 23 cleared games to 13. So ~32 points come from the harness alone, not model knowledge (the games also post-date the model's training cutoff, and an integrity audit checks for web-tool leakage).

Efficiency is the more striking result: on the 13 games all three world-model systems fully clear, Twin uses 3,357 scored actions vs. 5,367 (OPINE-World), 5,381 (EWM), and 7,485 for the human reference. Across its 23 cleared games it averages 0.61× the human action count, beating or matching humans on 21.

Does the twin actually generalize, or just memorize the log? Appendix A tests final twins on state–action pairs never observed (a visited state, an action never taken from it), grading against a fresh rerun of the real engine:

70.1% exact-frame prediction on unseen pairs — where one wrong cell in 4,096 fails the frame — is real rule induction, not replay. The failure modes are legible too: in the median game, 561 of 4,096 cells contain 90% of all error mass (typically a sprite or counter the twin models slightly wrong), and one game (wa30) exposes the memorization risk vividly: near-perfect replay, 9.9% generalization.

The goal side tells the opposite story. The first committed goal hypothesis is correct on 156 of 179 completed levels (87.2%) — impressively good. But graded per-action rather than per-level (Appendix J), the goal predicates achieve recall 0.771 at precision 0.214: they fire on 646 states, only 138 of which actually end levels. Per-game goal precision correlates at r = 0.77 with whether endgame plans succeed. Twin's two unfinished games are both goal failures with healthy dynamics: sp80 stalls at 92.3% dynamics accuracy because level 6's goal resists, and sc25 has a hidden countdown that charges a real cost for every goal-ordering hypothesis — trial-and-error Codex actually beats Twin there (44.8 vs 32.7), the one game where the ablation wins.

Cost: 2.60B processed tokens and 91.4 hours across the 25 runs, ~224k tokens per scored action — 4.7× the no-twin ablation's per-action cost, but 7.7× fewer total tokens than EWM while scoring 29.5 points higher. The twin front-loads compute offline so the online actions are few and correct.

What this changes, and what to doubt

The paradigm point is the interesting one. This is the WorldCoder recipe (LLM-synthesized transition/reward code, repaired from counterexamples) pushed fully online, against raw frames, with hidden controls and hidden goals — and the crucial engineering lesson is that the consistency check must be enforced by the harness, not requested by the prompt. That one hard gate appears to explain why Twin alone clears the long games (bp35, lf52, sk48: 9, 10, 8 levels) where prompt-level validation lets errors compound. It also relocates the LLM: instead of being in the decision loop every step, it constructs an artifact that removes it from the inner loop, with BFS running under machine verification. Test-time compute goes into model inference, not answer sampling. And the conclusion inverts intuition: building a usable world model of an unknown environment is "simpler than anticipated"; inferring the objective is the hard residue. If you believe goal inference is where lifelong human priors do their work, that's a pointed research direction.

Skepticism, in order of importance. First, the inductive prior is doing heavy lifting: ARC-AGI-3 games are deterministic, discrete, small-state, fully-observed-ish, and program-compressible — exactly the regime where exact replay is a free, decidable oracle. Continuous observations, stochastic dynamics, or truly latent state (a variable no frame ever shows) all break the exact-equality contract; the authors say so plainly in Limitations. The bp35 result (a scrolling camera hiding two-thirds of the map, cleared under the human action count by paying one probe per hidden strip and caching it) shows the approach stretches to hidden map, but hidden state remains out of scope. Second, this is one benchmark family, and the concurrent-systems comparison is muddy: OPINE-World and EWM use different base models, Prime Agent with Claude Opus 5 reports 95.5 (above Twin's 93.3), and Schema self-reports 98.98 via a best-of-two-models fallback. Twin's claim to the cleanest evidence is the fixed-base-model ladder (7.8 → 61.1 → 78.3 → 93.3), which genuinely isolates the harness — but "state of the art on ARC-AGI-3" is not settled by this paper. Third, the goal-discovery signals (color appears/disappears, local burst, big change) are generic visual heuristics but still a grid-game prior; whether an analogue exists for richer observation spaces is open. Fourth, 91 hours and 2.6B tokens for 25 games is not cheap, and compute concentrates pathologically on goal-hard games (ka59 alone: 24% of all tokens).

Where to spend your reading time: the Method section for the asymmetric-objective framing and Algorithm 1, then Appendix J — the per-action grading of goal claims (precision 0.214, false claims spread uniformly through levels rather than clustering near the finish) is the most honest and diagnostic analysis in the paper, and it's what turns "goal inference is hard" from a slogan into a measurement. Appendix K is worth skimming just to see the artifact: the complete ft09 twin, 305 lines of agent-written Python whose comments read like a lab notebook, is the most concrete picture I've seen of what a "legible world model" actually looks like.