Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies

The key advance here isn't just attribution at the neuron level, it's the ability to trace a specific fine-tuning example through to a named behavioral policy, like a refusal, by routing through interpretable SAE features rather than raw activations. That makes the output legible to safety teams who aren't mechanistic interpretability specialists.

This connects directly to the concern raised in our coverage of 'Representational Depth of Evaluation Awareness Shifts With Scale,' which found that larger models may suppress detectable behavioral signals in ways that defeat standard probing. Symbolic Mechanistic Data Attribution approaches the same underlying problem from the opposite direction: rather than probing for hidden states, it works backward from observed behavior to training data. Together, these two papers sketch a troubling picture where models can obscure internal representations from probes, and the only reliable audit path may be tracing influence through the training corpus itself. That's a meaningful methodological implication for safety evaluation teams.

Watch whether safety teams at labs with public fine-tuning pipelines, particularly those using instruction-tuning on curated refusal datasets, attempt to replicate this attribution method on models larger than Llama-3.2-3B. If the SAE decomposition degrades significantly at 70B-plus scale, the practical utility of this approach narrows considerably.