Representing concepts as functions — the research turn from finding features to trusting them
The interesting research question has moved. The first wave of interpretability asked whether interpretable structure could be found at all. The 2026 wave asks what the right representation of that structure is, and whether an adversary that learns can keep evaluation honest. Both are signs of a field growing up.
Sparse autoencoder neural operators represent a concept as a function over the input domain rather than a single activation value — capturing not just whether a concept is present but how and where it is expressed. That is a genuine expansion of what a feature can encode, and it is driven by the need to make features trustworthy enough to reason about.
Richer features because reliance demands them
A scalar feature answers a yes/no-ish question: is this concept active? A function-valued feature says where in the input it lives and how it varies — more surface to verify against than a single number that could match many internal states. If alignment is going to rely on features to catch deceptive and covert failures, the features need to carry enough information to be checked and steered precisely.
The evaluation side is turning the same way. Learning-based automated red-teaming replaces hand-written jailbreaks with a trained adversary that improves against its target — the evaluation-side answer to models that have learned what fixed evaluations look like.
The maturation pattern
Function-valued features, learned adversaries, weight-grounded explanations, reproducibility benchmarks — the common thread is a field moving from 'can we do this at all' to 'can we do it reliably enough to depend on.' That is the transition every method makes on its way from research curiosity to load-bearing infrastructure.
Finding a feature was the demo. Trusting it is the work.
arXiv — Mechanistic interpretability with sparse autoencoder neural operators → · arXiv — Learning-based automated adversarial red-teaming for LLMs →