Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals

The paper doesn't just benchmark uncertainty methods; it establishes that a simple frequency-based approach (bootstrap mode) works five times better than the field's current default (log-probability). That gap suggests the interpretability community has been using the wrong confidence metric without realizing it.

This connects directly to the Automated Benchmark Auditing work from this week, which exposed systematic flaws in how we measure AI systems. Activation oracles are one of the tools auditors use to inspect model behavior, but if their confidence scores are miscalibrated, auditors get false signals about what's actually happening inside the model. Reliable confidence is the prerequisite for trustworthy inspection. The WhoSaidIt collaborative annotation framework also touches this problem from a different angle: it treats disagreement as information rather than noise. Here, the problem is that oracles themselves are disagreeing with their own confidence levels, and fixing that disagreement is what unblocks their use in safety work.

If Qwen3.6-27B shows the same 5.7% calibration error improvement with bootstrap mode frequency on held-out mechanistic interpretability tasks (not just the benchmark used here), that confirms the method generalizes. If the improvement collapses on other model families or on adversarial inputs designed to fool frequency-based estimates, the result was overfit to Qwen's architecture.