Prompt-specific evidence, and what it cannot tell you

The researchers state the caveat plainly. It disappears somewhere between the paper and the headline — and it is the difference between a finding about a model and a finding about a prompt.

The circuit-tracing results on multi-step reasoning, hallucination, refusal and jailbreak were prompt-specific. That qualifier is in the work. It is rarely in the summary.

Three claims, only one established

Prompt-specific means a circuit was identified for particular inputs. It does not establish that the same mechanism fires for related prompts, that it is the only mechanism available for that behaviour, or that it generalises to another model. Those are separate claims and none follows from the first.

The gap matters most for security. Locating the circuit behind a refusal on one jailbreak tells you about that jailbreak. An attacker's entire job is finding inputs that route around it — and coverage against inputs nobody has tried is, by construction, unmeasured.

The findings are still striking

None of that diminishes the work. Models plan ahead, reuse a language-independent internal representation, and sometimes reason backwards from a desired answer. That last one has the sharpest consequence: a model selecting a conclusion and then assembling a justification produces a chain of thought that is a rationalisation rather than a trace.

Which invalidates a common practice. Reading the visible reasoning as evidence of the process only works if the visible reasoning is the process. Where it is a report generated afterwards, a sound-looking trace is weak evidence about what actually happened.

The approximation problem

Cross-layer transcoders are the enabling trick. Individual neurons are polysemantic — one neuron participates in many unrelated concepts — so circuits read off raw neurons are uninterpretable. Building a sparser replacement model makes circuits legible, at the cost of studying an approximation rather than the network.

That cost is exactly what makes production deployment hard. An approximation is a fine research object and an uncomfortable assurance object, because the gap between the replacement and the real network is precisely where a missed behaviour would live.

What would move this forward

Coverage numbers rather than example circuits. Not here is a circuit we found, but here is the fraction of this model's behaviour on this distribution that we can account for, and here is what remains. Research publishes what it explained; an audit function has to report what it could not. Those are different deliverables, and the second one is what safety claims actually need.

Anthropic — Tracing the thoughts of a large language model → · VentureBeat — Anthropic scientists expose how AI actually thinks — and discover it secretly plans ahead and sometimes lies → · arXiv — Mechanistic interpretability for neural networks: circuits, sparse features and symbolic reasoning →