Fine Until Fine-Tuned: Repeated Solutions Make Reasoning Fragile

TL;DR: The popular low-data reasoning recipes (s1, LIMO) work by showing a model the same ~500–1000 worked solutions many times over. This paper shows that repetition plants a time bomb: the drilled model looks identical to a normally-trained one at the end of training (~95% on held-out competition math), but a single, unrelated round of ordinary instruction tuning later drops it to 86%, and harsher follow-up stages take it to 59% or 22% — while a model trained on the same number of updates with unique solutions is untouched. A carefully constructed control shows the culprit is the repeated text, not the small number of problems, and the damage turns out to be suppression rather than erasure: five gradient updates of reasoning training restore nearly all of it. The whole study is pre-registered, with 70 numbered predictions and their outcomes published, which is worth attention in its own right.

The handoff illusion

The context: s1 and LIMO demonstrated that you can teach a base model to reason with a thousand or fewer curated chain-of-thought examples, compensating for the tiny dataset by training multiple epochs (s1: 5, LIMO: 15). Kopiczko et al. even found that 400 examples repeated many times beat 128× as many examples seen once. Judged when training ends — the paper calls this point handoff — repetition looks free or better than free.

But released models get trained again. Developers add an instruction-tuning stage; downstream users fine-tune on their own tasks. The question this paper asks is whether repetition changes what survives that next stage — something no handoff benchmark can reveal, by construction.

The answer is a clean, large, and non-obvious yes, and the design that establishes it is the best part of the paper.

Three arms, one variable

Start from Qwen3.5-9B-Base, which already solves 94.7% of the test set (MATH-500 levels 3–5 with integer answers, reasoning block force-opened) — so the reasoning training teaches it essentially nothing. That's deliberate: any later difference between models must come from how the solutions were presented, not what was learned. All arms train on the base model's own correct solutions, with identical hyperparameters: 140 updates, batch 32, LoRA rank 32, lr 10^{-4}. Only the texts differ:

• Drilled: 579 solutions, one per problem, cycled ~7.7 times each (the s1/LIMO regime).
• Once-trained: 4,480 different solutions, each seen once.
• Fresh-solution: the drilled model's same 579 problems, in the same order, visited just as often — but with a different correct sampled solution at every visit.

The third arm is the crux. It disentangles the two things that make drilling special: seeing few problems versus seeing the same token sequences repeatedly. If few problems were the issue, the fresh-solution model breaks too. If repeated text is the issue, it doesn't.

What breaks, and how badly

At handoff the three 9B models are within 1.8 points of each other in every math setting. Then each gets a second round of training on non-math data. The realistic stage: one epoch of instruction tuning on No Robots (8,544 everyday instructions) at the same learning rate as the reasoning training. Two stress tests: twenty updates at 3× the rate on instructions, or on short program-synthesis puzzles with bare answers as targets ("answer-only", which rewards responding without thinking).

The fresh-solution model survives every stage within a point of the once-trained one (all confidence intervals include zero). The drilled model's extra loss over the once-trained model is 9.6 points [6.9, 12.4] after one realistic epoch, 36.2 in the instruction stress test, 73.1 in the answer-only test. So the registered causal test passes decisively: it's the repeated token sequences.

Two important refinements. First, a measurement subtlety: instruction tuning makes every model — including the untrained base — stop opening its reasoning block when answering freely. That's a known, boring habit loss. So the paper always evaluates with the reasoning block force-opened; the drilled model's collapse is what remains after controlling for the lost habit. The drilled model reasons badly even when forced to reason: after the answer-only stage its median response is 103 tokens versus 1,294 for the once-trained model — it opens the block and gives up.

Second, this isn't a Qwen-plus-self-distillation quirk. Drilling long traces from a stronger model (gpt-oss-120b) — which is how s1 and LIMO are actually built — breaks worse: 33.7% after the instruction stress test, 1.1% after answer-only, with fresh traces again fully protective. The effect replicates at 35B, in Nemotron 3 Nano (a hybrid Mamba–Transformer from a different family), across three independent 9B runs, and on an arithmetic search task. The magnitude varies wildly across runs (extra loss from 12.6 to 61.1 points in the instruction stress test), which the paper is honest about: the claim is the direction, not the size.

Suppressed, not erased

Here's the result that reframes what the damage is. The broken drilled model (at 59.3%) is retrained on reasoning data — the base model's correct solutions to problems no model had seen. One update takes it from 59.3% to 84.7%. Five updates (160 examples) close 98% of the gap to the once-trained model.

More striking still: twenty updates on the base model's responses to non-math chat prompts — data that demonstrates the format of reasoning (open block, think, close, answer) with almost no mathematics — closes 97–98% of the gap too. The math is still in there. The later stage doesn't delete the capability; it makes it inaccessible. Notably, mere prompting doesn't unlock it: forcing the block open and nudging with "Wait" recovers less than half of the loss. It takes gradient updates, but astonishingly few.

The geometric intuition the paper's own analysis suggests (Appendix F): drilling narrows the model's reasoning onto a few sharpened paths. The later stage barely moves the once-trained model off its own solutions (0.0005 nats/token after two updates) but drives the drilled model far off its own solutions (0.067 nats/token) — under identical training losses on the new task. A small perturbation knocks the drilled model off a narrow ridge; a small push puts it back.

Can you see it coming? Sharpening is not the mechanism

Handoff accuracy gives no warning, but a distributional probe does. Define B(\theta) as the average extra negative log-likelihood the trained model \theta assigns to the base model's own forced reasoning, relative to the base — an estimate of the forward KL \mathrm{KL}(p_{\text{base}} \| p_\theta). The drilled model's B is 0.111 nats/token (base tokens ~10% less likely); the once-trained and fresh-solution models sit near zero. 63% of that divergence comes from positions where the drilled model keeps the base's top token but becomes more certain of it — "over-sharpening", the same direction self-training and RL push.

The obvious hypothesis is that over-sharpening is the fragility. The paper tests it directly with an elegant control: train exactly like the once-trained model (unique solutions, seen once) but on solutions sampled at temperature 0.5, which concentrates on the base's likely choices without repeating any text. This sharpened model reaches B = 0.082 — three-quarters of the drilled model's — and comes through both stress tests unharmed (extra loss under 2 points, intervals including zero). So B flags drilled models cheaply (one forward pass per sample) but the repeated text matters beyond the sharpening it causes. Caveats the paper itself raises: a threshold in B between 0.082 and 0.111 isn't excluded, and on the arithmetic task there's a counterexample model with tiny B that still broke. B is a smoke detector, not a mechanism.

Remedies, and the edges of the claim

Two fixes work completely. Fresh solutions: sample a new correct solution at every visit — costs 8 correct samples per problem instead of 1, but zero extra problems, and prevented the damage everywhere it was tried. Replay: mixing just 6.25% of the original reasoning texts into the later stage cut the drilled model's extra loss from 13.1 points to 1.1 (tested only in a gentle stage). So the claim is scoped to later training without replay — exactly the situation of a downstream user fine-tuning a released model on their own data.

The other boundary: everything above concerns reshaping a skill the base model already has (Qwen3.5-9B is near ceiling on this math), which is precisely the regime s1 and LIMO argue for. On a genuinely new skill — five-digit base-7 multiplication within a 2,048-token budget, which the base solves at 0.2% — repetition's cost showed up at handoff instead: the drilled model learned to 37.9% versus 66.3% once-trained. There, what was learned was fragile regardless of recipe.

How much to trust it

Strengths: the design isolates one variable cleanly; every experiment was registered in advance with numerical thresholds (70 predictions, 50 held, 17 failed — all listed, including the failures); an independent audit recomputed the main results; total compute was ~$2,400. The failures are reported where they qualify claims — e.g., the drilled-on-traces model missed a predicted 10-point threshold in a gentle stage (6.9 points).

Weaknesses to hold in mind. Everything is LoRA (rank 32, replicated at rank 128); full fine-tuning is untested, and Biderman et al. showed LoRA both learns and forgets less — the five-update recovery in particular might not transfer, since LoRA never touches base weights. The realistic later stage is one dataset at one learning rate; the "stress tests" at 3× the rate are deliberately harsh, and note that 60 gentle updates matching the stress test's lr×steps cost only 13.1 points versus 36.2 — the damage depends superlinearly on the later stage's learning rate, not just total step size. Effect sizes vary almost fivefold across runs. The break sets in somewhere between 3.3 and 7.7 passes on the 9B but earlier on other models, so "how many epochs is safe" has no clean answer. And training on gpt-oss-120b's traces moved the student only slightly toward the teacher (held-out NLL fell 1.2%), so this tests repetition of a stronger model's texts, not a full s1/LIMO reproduction where real learning occurs.

The practical upshot for anyone doing SFT-for-reasoning: evaluate after the next stage, with reasoning forced as well as free; if you must revisit problems, sample fresh solutions per epoch (cheap if you're distilling anyway); if you're shipping a drilled model, expect downstream fine-tuners to silently break it unless they replay a sliver of your data — or budget ~160 reasoning examples for repair.

If you read two sections, read Section 4 (suppressed-not-lost, the most surprising result) and Section 5 (the sharpening control, the cleanest mechanism experiment). Appendix F is the place to go if you want to think about why — the observation that instruction tuning drives the drilled model off its own solutions until they're less likely than under the untrained base, while simultaneously moving it slightly toward the base's distribution without recovering the base's competence, is the genuinely open puzzle the paper leaves behind.

One meta-note worth flagging: per the AI-use statement, the code and prose were written by an AI (Claude) under close human direction, with the humans owning every research decision. Combined with the pre-registration discipline and the frozen-output number registry, this is one of the more interesting existence proofs to date of rigorous, auditable AI-assisted empirical research — arguably as notable as the finding itself.