The Signs Were Always There: Training-Free Concept Detection and Steering in Raw Transformer Dimensions

TL;DR: This paper claims that the interpretability community has been solving a rotation problem that doesn't exist. Instead of training sparse autoencoders to find the "right" feature directions in transformer hidden states, the author argues you can read concepts directly off the raw coordinate axes: the sign of each dimension encodes content, the magnitude encodes strength, and a concept is just a subset of dimensions with a consistent sign pattern. With zero training, sign-agreement counting detects 175 semantic categories at AUC 0.97–0.99, beats a trained Gemma Scope SAE feature-for-feature, and — via a second, nearly disjoint "write catalog" derived from the unembedding matrix — steers concepts into fluent generated text on four language models. If this holds up (single author, not peer-reviewed, and there are important caveats about what "feature" means here), it undercuts the core premise motivating dictionary learning in interpretability.

The problem: everyone assumes you need a rotation

The dominant story in mechanistic interpretability goes like this. Transformers represent more features than they have dimensions, so features get packed into non-orthogonal directions (the superposition hypothesis, Elhage et al. 2022). Individual neurons and dimensions are therefore polysemantic — a single coordinate means many things at once — and to recover the true features you must learn a rotation: train a sparse autoencoder (SAE) on millions of contextual activations, at significant GPU cost, once per model per layer. Probes and steering vectors likewise require labeled data and training, one concept at a time.

There's a known crack in this story: Elhage et al. (2023) showed the residual stream has a privileged basis — despite being theoretically rotation-invariant, it empirically develops axis-aligned structure (outlier dimensions), possibly an artifact of Adam's per-parameter normalization. This paper drives a truck through that crack. Its claim is not just that the standard basis is privileged, but that it is sufficient: a full feature catalog for detection, and a second one for steering, are already sitting in the raw coordinates, readable by counting.

The key idea: sign is content, magnitude is volume

Picture each of the D residual-stream dimensions (2560 for a 4B model) as a one-bit register with a volume knob. The bit — is h_l[d] positive or negative at layer l? — says what the dimension is asserting. The magnitude |h_l[d]| says how loudly. A feature like "animal" is then a subset of dimensions \mathcal{D}_c together with an expected polarity pattern \boldsymbol{\pi}_c \in \{+1,-1\}^{|\mathcal{D}_c|}: tokens for animals reliably have dims {47, 512, 1893, …} positive and {203, 678, …} negative. To score a token, count the fraction of registered dimensions whose sign matches. No weights, no dot products with learned directions — normalized Hamming agreement.

Three preliminary results make this plausible. First, sign alone carries most of the prediction: setting every magnitude to 1 and pushing \text{sign}(\mathbf{h}) through the LM head preserves 60–93% top-5 next-token accuracy across four LMs (random sign permutations: 0%). Second, a fully learned-parameter-free version — majority sign over context embeddings, matched to vocab tokens by Hamming distance — narrows a ~248K vocabulary to the right 4096 candidates 80–90% of the time. Third, dimensions are functionally independent: pairwise mutual information between dim signs is under 0.006 bits everywhere (context lowers it), and an MLP with full cross-dimension capacity adds literally zero AUC over per-dimension reading. Whatever higher-order structure exists carries no information useful for feature detection.

The mechanism: a type cache, two catalogs, and a thermostat

The pipeline is almost embarrassingly cheap. Run every vocabulary token through the model alone (one token, no context, ~20 GPU-minutes total) and cache the hidden state at a chosen layer — since there's no context to mix in, this captures pure type identity as processed by the full stack. Store only sign bits: 93 MB covers Qwen3-32B's whole vocabulary at one layer.

Discovery. For a category with ~50 anchor tokens, compute for each dimension how well its sign separates anchors from the rest of the vocabulary (a per-dim AUC), register every dimension above threshold \tau=0.75 with its winning polarity, done. Null calibration with random anchor sets sets the reporting bar. Unsupervised variant: seed with a random token, find its 20 nearest sign-neighbors, keep dims where all agree — 1500/1500 random seeds yield valid sparse features (firing on <0.1% of vocabulary).

The write catalog. Here's the paper's most genuinely novel move. Flipping a concept's read signs away from expected suppresses it, but forcing them toward expected does nothing — the read dims already carry the right signs. Detection and production live on different dimensions. The write target for concept c is

the per-dimension sign consensus of the unembedding rows over a few seed tokens \mathcal{S}_c — a matrix-row sum, no forward pass, no gradients. This target is nearly disjoint from the read catalog: rank correlation between read strength and write polarity is 0.01–0.11 across models, and the top write dimensions detect at chance (read-AUC ≈ 0.55–0.61).

Steering. The write target is injected not into the residual directly but into the input of the attention output projection W_O — the same site as Inference-Time Intervention, but with the direction read off frozen weights instead of trained probes. Select the top-K coordinates whose W_O column signs agree with \boldsymbol{\tau}_c (dropping "generic amplifier" coordinates shared across concepts), then push only the currently sign-disagreeing coordinates, and only up to a margin. On top of that, a proportional controller reads concept presence p(t) (residual sign agreement with \boldsymbol{\tau}_c) each token and backs the gain off as the concept appears — a thermostat that prevents collapse into a repeated concept token. Decoding stays clean: greedy, no repetition penalty.

The evidence

Detection. Across Qwen 3.5-4B, Gemma 3-4B, Mistral 7B, and Qwen3-32B, prototype-level AUC on 175 curated categories is 0.975–0.993, and all 175 clear null calibration on every model. A trained logistic probe on the same dims reaches 0.9997 — and crucially, its weights converge to axis-aligned voting (99.9% sign agreement with the prototype polarities). Randomly initialized models: 0/175 categories detectable, so this is a product of training, not architecture or method artifact.

Versus an actual SAE. The head-to-head with Google's Gemma Scope 2 (16K features, layer 25) is the sharpest result:

Causality. Flipping a concept's registered signs (magnitudes untouched) during a live forward pass drops the concept's target logits by 5–24; forcing the same dims toward expected signs, flipping a disjoint concept's coalition, or flipping random dims all do roughly nothing. The effect requires flipping ~200–500 dims — the coalition, not any single dimension, is the causal unit.

Steering. On 12 concepts × 4 held-out prompts per model, scored by a deterministic LLM judge (concept present and at least as fluent as the unsteered baseline): Gemma 92%, Mistral 75%, Qwen3-32B 77%, Qwen 3.5-4B 62%. The qualitative demo is compelling: the fixed prompt "Yesterday I went to the…" continues to "supermarket to buy some food," "hospital to see my mom. I was so sad," or "doctor for a check up on my knee" depending only on \boldsymbol{\tau}_c.

Cross-modality. The same recipe finds per-dim sign structure in DINOv2 (self-supervised vision, 9/12 ImageNet superclasses above 0.70), ViT-Base (supervised, 11/12), and AST (audio, all 50 ESC-50 categories, 47/50 above 0.80), with MI just as negligible and MLPs just as useless. Supervision sharpens the structure but doesn't create it — the paper's inference is that this is a convergent property of gradient descent on residual-stream architectures.

On superposition. The paper replicates Elhage's toy autoencoder, confirms superposition emerges there, and shows sign matching still recovers 80–100% of the toy's features. But the toy's bottleneck regime has cross-dim sign MI of 0.05–0.10 bits — 10–50× what real transformers show — and the toy's MI drops to real-transformer levels as the bottleneck relaxes. The suggestion: geometric interference is a function of bottleneck pressure, and real models may not operate in the regime where that pressure binds.

What to be skeptical about

This is a single-author arXiv paper making sweeping claims, so calibrate accordingly. Several specific concerns:

These are lexical features. Everything is anchored to a single-token type cache: the 175 categories are things like "animal," "metal," "conjunction" — vocabulary-level categories. Flagship SAE features are contextual and abstract: sycophancy, code bugs, the Golden Gate Bridge across languages and images. The SAE head-to-head is run on the paper's home turf (token-category detection), where a type-level method is structurally advantaged. The polysemy test (§3.4) partially rebuts "it's just token identity" — the same prototype scores "train" the vehicle at 0.93 vs. 0.45 for "train the network," 77–80% accuracy over 77 cases on three models — but Qwen3-32B stays near chance, and this reads cross-category sense shifts, not fine-grained meaning.

"+0.018 AUC" undersells the probe. 0.9997 vs. 0.9814 is a ~60× difference in error rate. For high-stakes monitoring that gap matters; the honest framing is "1-bit quantized, axis-aligned, threshold-selected weights get you almost all the way," which is still striking but not "learning buys nothing."

"No rotation" is partly semantic. A sign-agreement score over ~200–900 registered dimensions is a linear readout — with ternary weights (\pm 1 on registered dims, 0 elsewhere). The genuine finding is that axis-aligned, 1-bit-quantized directions suffice and that trained probes converge to them. That's a real constraint on the geometry (and consistent with the privileged-basis literature), but "dimensions as binary registers read one at a time" oversells it, especially since causal effects require coalitions of hundreds of dims.

Steering evaluation is thin. 48 trials per model, one LLM judge, no human evaluation, and no head-to-head against ActAdd/CAA/ITI on the same concepts — the natural baseline comparison is conspicuously absent. Suppression is validated only at the logit level; fluent suppressed generation is explicitly future work. Hyperparameters (write band, coalition size, basin depth, thermostat ceiling) are hand-set per model, and reasoning-tuned models needed their chat template as a "fluent carrier."

Scale and D/V. Per-dim sharpness tracks dimension-to-vocabulary ratio, not parameter count, and the biggest model tested (32B) is the weakest on the contextual reads. Whether this survives at 70B+ with large vocabularies is open.

Why it's worth your attention anyway

Even discounted, two things stand out. The read/write dissociation — that the dimensions whose signs detect a concept are nearly disjoint from those that produce it, with the write target readable off frozen unembedding weights — is a clean, cheap, novel object, and it explains a real puzzle (why forcing "toward" signs never induces concepts, and why early single-layer sign-flip studies found nothing: corruption deep in the stack gets re-derived downstream). And the MI evidence against practical superposition in real residual streams (<0.006 bits pairwise, MLPs adding nothing, in language and vision and audio) is a concrete, falsifiable challenge to the assumption that motivates the entire SAE program. If the field's expensive dictionaries are mostly learning a rotation that buys +0.018 AUC over quantized axis-aligned reading, that's a result someone should replicate immediately — and replication is easy, since the whole pipeline is a type cache and some sign counting.

Where to spend your time: §3.5–3.6 (the write catalog and closed-loop steering — the most original material), Appendix A (the SAE head-to-head), and the superposition discussion in §5 with Appendix F. If you only verify one thing, verify Table 8's toward/away asymmetry — it's the hinge the whole read/write story turns on.