Activation probes detect rare AI misalignments with minimal labeled examples

The paper's key insight isn't just that probes work with few examples, but that imbalanced training (many negatives, few positives) outperforms balanced datasets in this regime. This inverts conventional ML wisdom and has direct implications for how safety teams should structure their labeling workflows when misaligned behavior is genuinely rare.

This research directly addresses the monitoring gap exposed in the OpenAI audit from late October, which found that 25 percent of frontier model traffic operates without detection. The sparse-label finding matters because it removes a major excuse for that gap: safety teams can no longer claim they need balanced datasets or abundant positive examples to train reliable detectors. The work also builds on the linear probe transfer results from late October, which showed probes remain effective across model variants. Together, these papers suggest a practical path toward the real-time monitoring architecture OpenAI described in their own safety announcement, though the audit's redaction prevents us from knowing whether OpenAI is actually using these techniques.

If OpenAI or Anthropic publishes details about their actual probe training regimes in the next six months and confirms they're using imbalanced negative-heavy datasets, that validates this paper's recommendations for production. If instead they report using balanced or synthetic upsampling approaches, it signals either that the lab's threat model differs from the researchers' assumptions or that the sparse-label regime doesn't survive contact with real deployment constraints.