Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?

The study's sharpest contribution isn't just that open and closed models diverge internally, it's that the divergence is invisible at the output layer, meaning the standard validation signal practitioners rely on (matching predictions) is precisely the signal that fails to catch the problem.

This finding sits largely disconnected from the recent Anthropic policy coverage on the site, including the Fable 5 reinstatement and Mythos access restoration reported by TechCrunch and The Verge around July 1. Those stories concern deployment gating and regulatory clearance, not interpretability methodology. The relevant thread here is upstream: as frontier closed models like GPT and Gemini become more central to safety audits and alignment research, the tooling used to interrogate them matters enormously. If surrogate-based interpretability produces structurally misleading attribution maps, then safety arguments built on those maps carry hidden fragility, regardless of whether the underlying model is commercially available or policy-restricted.

Watch whether safety teams at labs using API-only interpretability pipelines, particularly those building red-teaming or audit infrastructure around GPT or Gemini, publish methodological updates that address surrogate selection criteria within the next two quarters. Silence there would suggest the field hasn't absorbed this result.