Geometric Evolution Maps: Extracting Stable Concept Probes from Transformer Residual Streams

The deeper issue GEMs surfaces is that most published interpretability findings built on concept probes may have been measuring representations mid-rotation rather than at their settled form, which quietly undermines a body of prior work rather than just improving future studies.

This connects directly to the same-day coverage of 'Quantization Benefits of Residual-Free Transformers,' which identified residual connections as an active shaping force on internal representations rather than a passive routing mechanism. Both papers, arriving the same week, push toward the same uncomfortable conclusion: the residual stream is doing more structured, layer-dependent work than standard practice assumes, and tooling built without accounting for that structure is unreliable. For interpretability specifically, GEMs matters because safety and debugging workflows depend on probes that actually reflect stable model knowledge. If those probes were sampled from the wrong layers, the diagnostic value of prior audits is genuinely uncertain.

The real test is whether established interpretability benchmarks, particularly those used in safety evaluations, show meaningfully different probe accuracy when re-run with GEM-identified settlement layers versus the arbitrary late-layer defaults. If major interpretability labs (Anthropic, DeepMind) adopt GEMs in published audits within the next six months, that signals the field accepted the critique; silence suggests the methodology gap is being quietly absorbed rather than corrected.