'Capturing Polysemanticity with PRISM' arXiv 2506.15538 — multi-concept feature description framework addresses polysemantic SAE features
The PRISM arXiv paper (2506.15538) introduces a multi-concept feature description framework specifically addressing polysemantic features in sparse autoencoders. Single-concept feature descriptions (current standard) fail when features genuinely encode multiple meanings; PRISM's multi-concept framework captures the polysemanticity that single-concept methods reduce to single-meaning approximations.
The substantive piece is the polysemanticity-capture methodology refinement. Pre-PRISM SAE interpretability assumed (often implicitly) that sparse autoencoder features encoded single meanings — the 'monosemanticity' goal. Empirical findings showed many features were polysemantic (encoding multiple distinct meanings). Single-concept description methodology forced these features into single-meaning approximations that lost interpretive precision. PRISM's multi-concept framework addresses the methodological gap.
The competitive read against DeepMind's SAE deprioritization is that methodology refinements like PRISM may address some of the limitations that motivated the deprioritization. If polysemanticity-capture improves SAE-feature interpretability accuracy at safety-relevant tasks, the methodology refinement could close part of the baseline-underperformance gap that DeepMind cited.
arXiv — Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework → · arXiv — Residual Stream Analysis with Multi-Layer SAEs →