Interpretability-Guided Layer Selection over Subspace Projection: SAEs as Stethoscopes, Not Scalpels, for Raw Task Vector Model Editing

The deeper implication isn't just that SAEs fail at this task: it's that the gap between understanding a model's internal representations and actually steering its behavior may be structurally larger than the interpretability field has assumed. Diagnostic clarity and interventional power are not the same capability, and this paper puts empirical numbers on that distinction for the first time in the task-vector editing context.

This connects directly to the ACROS work covered the same day ('Sense Representations Are Inducible Interfaces'), which took the opposite approach: rather than projecting onto existing internal structure, it injects a gated residual pathway to add steerable semantic representations without touching base weights. That method sidesteps the energy-loss problem entirely by not relying on SAE subspace fidelity. Read together, both papers point toward the same practical conclusion: effective model steering likely requires additive or bypass architectures, not projection onto learned feature bases. The activation steering piece from the same day's coverage adds a third data point, showing that even parameter-efficient steering methods carry their own fidelity tradeoffs when diversity is measured.

Watch whether follow-up work tests hybrid approaches that use SAE features diagnostically to select which layers to target, then apply raw task vectors at those layers. If that combination recovers meaningful performance on mathematical reasoning benchmarks within the next two conference cycles, it validates the 'stethoscope' framing as a practical workflow rather than just a rhetorical reframe.