Sparse autoencoders spread beyond language models — into speech recognition
An SAE has been trained on frame-level embeddings from Whisper's encoder, learning a high-dimensional sparse latent space over a Transformer-based speech recognition model. The technique developed to untangle language-model activations is being carried into a modality with different structure and different failure modes.
The transfer is a genuine test of the method rather than an application of it. SAEs assume that activations are sparse combinations of interpretable features in superposition. That assumption was developed against text. Speech has continuous time structure, speaker variation and acoustic noise that text does not, so whether the same decomposition yields monosemantic features says something about whether the assumption is about transformers or about language.
There is practical value regardless of the theoretical answer. ASR failures are notoriously hard to diagnose — a model mistranscribes an accent or a noisy segment and the error is opaque. Feature-level inspection offers a route to saying which learned feature fired instead of the right one, which is the difference between a bug report and a shrug.
Related work on cosine-scored SAEs and on emergent misalignment through feature superposition geometry suggests the method itself is still moving. That is worth flagging plainly: results obtained with one SAE variant do not automatically hold for another, and the field has not settled on which variant to trust.
arXiv — On the interpretability of Whisper encodings using sparse autoencoders → · arXiv — Size doesn't matter: cosine-scored sparse autoencoders → · arXiv — Understanding emergent misalignment via feature superposition geometry →