Mapika/decider

TL;DR: decider is an open reproduction of the "System One" model class (TypeSafe AI's Jev): language models that never generate text, instead answering a batch of typed questions — choices, scores, yes/no — over a state in a single forward pass, returning calibrated probability distributions with no decoding or parsing. It's the most complete of the ~15 such projects that appeared this week: 0.8B/2B/35B-MoE weights on the Hub, full training code and data mixture, an RL calibration stage, browser-agent evals, and unusually honest reporting of where it loses.

What it is and why it matters

The pitch: agent stacks are full of "which tool? which team? is this safe? how frustrated is the user?" decisions currently implemented by generating JSON and parsing it, or by constrained decoding one token at a time. Decider reframes these as typed questions answered in one pass, each with a proper probability distribution — so you get confidence gating, abstention via catch-all options, and composable probabilities (the examples include a hierarchical beam over Choice probabilities) essentially for free. Three question types: Choice (2–255 options with optional descriptions/rubrics), Score (2–10 described levels, returns the expected level), Noul (P(yes)). State is a string or arbitrary JSON up to 32k tokens.

Two properties distinguish it from prompted-LLM-plus-logit-parsing: independence (each question scored on its own row, so reordering questions can't change answers — packed prompts flip up to 12% of answers) and isolated Score levels (each level judged without seeing its neighbors, then normalized; before training for this, the fits summed to 1.4–3.5).

How it works

The prompt renderer places one answer slot per question; the model reads the hidden state at each slot, projects onto one label token per option, and softmaxes over valid labels. Nothing is generated, so all questions come out of one forward.

The clever engineering is the schema cache. A second trained layout puts the questions before the state, making the whole schema a state-independent prefix. That prefix's cache — K/V for the 6 full-attention layers, conv and recurrent state for the 18 delta-net layers, held read-only — is computed once; each request then runs only the state plus slots as a CUDA graph. A 151-option classifier drops from 217 ms to 11.5 ms per batch-32 forward (19×); 10-question chat routing hits ~11,180 decisions/s on one GH200. State-first (the default, more accurate) gets the mirror-image trick: run the state once, fork its cache to every question.

Evidence

On the 95-task regression set: 2B v10 at 0.805 in-task / 0.755 held-out accuracy (ECE 0.037/0.084); 35B-A3B (3B active, experts frozen, Muon, one epoch, no RL) beats it on 93 of 95 tasks at 0.855/0.810. The v10 RL stage (proper-score belief reward on live browser tasks and exact games) took MiniWoB++ sampled play from 83% to 93% and cut belief divergence from exact game laws from 0.47 to 0.22 nats — notably, the 35B without RL wins on greedy play but loses on sampled play, a clean demonstration that argmax accuracy and served-distribution quality are different things.

Against the closed reference: 35B matches Jev 1.13.0 on Bespoke's suite (0.774 vs 0.760 macro) but trails on JevBench hard (0.676 vs 0.730); the 2B sits at 0.459 there with top-label ECE of 0.30 — confidently wrong on multi-hop and long-policy items.

Caveats

English only. The v10 RL stage is not reproducible from this repo (needs a separate research codebase). Schema-first costs 1.5 points on fixed-label tasks and 5–24 on dynamic options and long states. Small models don't follow rules written into questions (form-filling: 0.67 with a plain question, 0.24 with spelled-out rules — a 0.7M specialist scores 0.9995). Teacher labels come from a 27B Qwen that agrees with only 72% of its own generic-option labels. The vision variant lags on v5 weights. And per the selection note: this ecosystem is days old — before adopting, verify calibration on your own labels (the README says exactly this) and benchmark against constrained decoding; decider's real edge is per-question caching throughput and calibrated distributions, not raw accuracy.

Try it

pip install git+https://github.com/Mapika/decider

from decider.infer import Decider
d = Decider("Mapika/decider-2b")  # ~4 GB bf16, one CUDA GPU
d.decide("My card was charged twice.",
         [{"question": "Which team?", "options": ["billing", "technical", "sales"]}])
# [{"choice": "billing", "confidence": 0.77, "probs": {...}}]

Serving speaks TypeSafe's wire format (scripts/serve.sh, point TYPESAFE_BASE_URL at it); the vision variant has a browser demo Space linked from the model table.