Circuit tracing moves from paper toward production safety
Attribution graphs and cross-layer transcoders — built to work around the polysemantic nature of individual neurons — are being carried out of research and into production monitoring. The underlying findings remain striking: models plan ahead, reuse a language-independent internal representation, and sometimes reason backwards from a desired answer.
Reasoning backwards from a conclusion is the finding with the sharpest safety consequence. A model that selects an answer and then assembles a justification produces a chain of thought that is a rationalisation rather than a trace — which invalidates reading the visible reasoning as evidence of the process that produced it.
Cross-layer transcoders are the enabling trick worth understanding. Individual neurons are polysemantic — one neuron participates in many unrelated concepts — so circuits read off raw neurons are uninterpretable. Building a replacement model with sparser structure makes circuits legible, at the cost of studying an approximation rather than the network itself.
That cost is exactly what makes the move to production hard. An approximation is a fine object for research and an uncomfortable one for assurance, because the gap between the replacement model and the real one is where a missed behaviour would live.
Anthropic — Tracing the thoughts of a large language model → · VentureBeat — Anthropic scientists expose how AI actually thinks — and discover it secretly plans ahead and sometimes lies → · Subhadip Mitra — Circuit tracing for the rest of us: from probes to attribution graphs →