'Beyond Activation Patterns: A Weight-Based Out-of-Context Explanation of Sparse Autoencoder Features' arXiv 2601.22447 — methodology paper proposes weight-based SAE feature explanation alternative to activation-pattern analysis
The arXiv 2601.22447 paper proposes weight-based out-of-context explanation methodology for sparse autoencoder features — an alternative to the activation-pattern analysis that dominates SAE interpretability research. The weight-based methodology extracts feature explanations from autoencoder weights directly rather than from observed activations on input distributions.
The substantive piece is the activation-vs-weight methodology bifurcation in SAE interpretability. Activation-pattern analysis (current standard) requires running the model on input distributions to observe when features activate, then inferring meaning from the activation patterns. Weight-based analysis extracts meaning directly from autoencoder weights — independent of any specific input distribution. The methodological independence has implications for out-of-distribution interpretability.
The competitive read against the broader 2026 mech-interp methodology landscape is that the field is now investing in multiple methodology families simultaneously: activation-pattern SAE analysis (mainstream), causal-steering with concept vectors (Anthropic emotion vectors), multi-layer SAE methodology (residual stream analysis), domain-specific SAE applications (code correctness), weight-based feature explanation (this paper). The methodology pluralization addresses DeepMind's SAE deprioritization concerns by exploring whether methodology refinements close the baseline-underperformance gap.
arXiv — Beyond Activation Patterns: A Weight-Based Out-of-Context Explanation of Sparse Autoencoder Features → · arXiv — Residual Stream Analysis with Multi-Layer SAEs →