Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoders
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
Sparse autoencoders have become central to mechanistic interpretability work, but their reliability hinges on a largely unexamined problem: feature reproducibility across training runs. This paper quantifies that instability through a per-feature stability metric, revealing a critical divide. Stable features encode genuine predictive signal and drive reconstruction quality, while unstable features are noise artifacts triggered by low-frequency surface patterns. The finding reshapes how researchers should weight SAE discoveries, suggesting many published interpretability claims rest on spurious, non-reproducible features. For labs building interpretability pipelines or relying on SAE-derived insights for alignment work, this establishes a filtering mechanism to separate robust mechanistic findings from statistical flukes.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The deeper provocation here isn't just that some SAE features are noisy: it's that the field has been publishing mechanistic claims without a standard for checking whether those claims survive a different random seed. This paper is essentially proposing the first quality filter for a body of literature that has been accumulating without one.
The related coverage from this period is largely disconnected from this story. The nD-RoPE work and the sea surface temperature forecasting paper both touch on representational geometry and dimensionality, but neither engages with interpretability pipelines or SAE training dynamics. The more relevant context sits outside the current archive: the broader mechanistic interpretability push at labs like Anthropic, which has treated SAEs as a primary lens for understanding residual stream features. That body of work is what this paper implicitly audits.
Watch whether Anthropic or any lab with a public SAE release applies this stability metric retroactively to their published feature dictionaries and reports what fraction of claimed features survive the filter. If that number comes in below 50 percent, it will force a significant reassessment of which prior interpretability findings are worth building on.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsSparse Autoencoders · Neural Network Interpretability
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.