Sparse autoencoders extend to ASR with Whisper — arxiv:2605.12225 trains high-dimensional sparse latent on frame-level embeddings, cross-modality interp goes mainstream
An arXiv paper (arXiv:2605.12225) applies sparse autoencoders to Whisper, OpenAI's Transformer-based automatic speech recognition system, training a high-dimensional sparse latent space on frame-level embeddings. The methodology extension marks sparse-autoencoder interpretability moving from text-only into the multi-modal/audio domain — the same toolkit Anthropic uses on Claude is now being applied to ASR systems with comparable methodology.
The methodology-extension substance is the substantive piece. Through 2024-2025 the dominant sparse-autoencoder interpretability work concentrated on text-based large language models — Anthropic's microscope methodology and the broader mechanistic-interpretability research operated almost entirely on the transformer-encoder/decoder text-token substrate. The arXiv:2605.12225 paper extends the methodology to the ASR substrate: frame-level audio embeddings from Whisper, treated as the feature substrate that sparse autoencoders learn over. The empirical result is that the cross-modality methodology transfers: features can be identified, polysemantic decomposition works, and the same interpretability primitives that operate on text apply to audio embeddings.
The methodology-consequence is the cross-modality interp generalization. Anthropic's microscope methodology for tracing model reasoning paths is the text-substrate baseline. The Emergent Misalignment paper's feature-superposition framing is the underlying-geometry framing that the cross-modality work extends. The combined picture is that mechanistic-interpretability tooling is generalizing past text to encompass audio, vision, and (eventually) multimodal frontier-model substrates. The deployment-research consequence is that safety-evaluation methodology that operates on feature-level analysis becomes applicable across model modalities rather than text-only, broadening the evaluation surface meaningfully.
ArXiv — Mechanistic Interpretability of ASR models using Sparse Autoencoders arXiv:2605.12225 → · JMIR AI — Sparse Autoencoders Enhance Mechanistic Interpretability LLMs Medicine 2026 → · IntuitionLabs — Understanding Mechanistic Interpretability AI Models LLMs →