Neuron Populations Exhibit Divergent Selectivity with Scale

The paper's deeper provocation is not just that specialization increases with scale, but that this pattern is predictable enough to follow a law, meaning interpretability researchers may now have a quantitative handle on how internal representations reorganize rather than treating it as opaque emergence.

This connects directly to the compression and redundancy thread running through recent coverage. The SubFit paper from June 1 argued that redundancy clusters unevenly across submodules, and the Rosetta Neurons finding offers a complementary explanation for why: as scale increases, a shrinking but increasingly monosemantic core of neurons may be doing more of the load-bearing work, while the rest of the network fills other roles. That framing also has implications for the multi-domain RL interference paper from the same day, which found that overlapping computational pathways cause cross-domain collapse. If specialized neurons consolidate at scale, the geometry of those overlaps likely shifts in ways that existing interference models do not account for.

The key test is whether this specialization trend holds across architecture families beyond the models Dravid et al. sampled. If a replication on mixture-of-experts architectures shows the same monosemanticity scaling curve, the finding is structural rather than an artifact of dense transformer training dynamics.