The transcript is not the thought
Chain-of-thought monitoring became the most widely deployed interpretability technique in the industry because it is legible. The evidence says legibility was the wrong thing to select for.
A 2026 paper argues latent-state dynamics, not the chain of thought, should be the default object of study for LLM reasoning — and earlier work found models mentioned the hints they actually used a minority of the time: about 25% for one frontier model, about 39% for another.
Why we believed the transcript
Chain of thought arrived looking like a gift. The model writes out its reasoning, in English, in order, and a person can read it. After years of arguing that neural networks were opaque, here was a system that appeared to narrate itself.
Nobody chose it because there was evidence the text corresponded to the computation. It was chosen because it was the only interpretability method that required no special access, no tooling, and no expertise to read.
We did not select for correspondence. We selected for readability, and got what we selected for.
What the numbers mean
A model that mentions its operative hint 25% of the time is not lying in the remaining 75%. It is producing text that is plausible given the question — which is what it was trained to do — while the computation that determined the answer happened elsewhere.
For oversight this is close to a worst case. An unreadable system invites appropriate caution. A readable system that is only sometimes faithful invites confidence that is wrong in an unknown fraction of cases, and provides no signal for which.
The awkward position of the field
Both things are true at once: real progress in tracing features and mapping paths from prompt to response, and a headline result that the transparency most of the industry actually relies on was never transparency.
The recommended direction — study latent states, instrument activations — is correct and expensive. It requires model access that almost nobody evaluating a model has, which pushes interpretability back inside the labs at precisely the moment external evaluation is becoming a regulatory expectation.
What to do on Monday
Keep the chain-of-thought monitoring. It catches real problems and costs almost nothing. Stop describing it as an assurance mechanism, and stop putting it in documents that make safety claims.
It is a smoke detector. Useful, cheap, worth having in every room — and not a fire suppression system, however much we would like it to be.
arXiv — LLM Reasoning Is Latent, Not the Chain of Thought → · MIT Technology Review — Mechanistic interpretability: 10 Breakthrough Technologies 2026 →