Non-linear Interventions on Large Language Models

Most interpretability and steering work to date has assumed that concepts inside LLMs are encoded as linear directions in activation space, a convenient simplification that makes the math tractable but may not reflect how models actually store complex or context-dependent behaviors. This paper's refusal bypass result is the concrete stress test: if non-linear steering outperforms linear methods on a safety-relevant task, the assumption was doing real work, not just theoretical tidying.

The related coverage on this site is largely disconnected from this paper's core contribution. The Crys-JEPA piece from the same day touches on multi-objective optimization in generative models, and there is a loose structural parallel in that both papers push against a simplifying assumption (likelihood maximization there, linearity here) that was constraining a field. But the interpretability and mechanistic safety literature this paper belongs to has not been a primary focus of recent Modelwire coverage, so readers should treat this as an entry point into that thread rather than a continuation of one.

Watch whether safety teams at major labs (Anthropic's interpretability group is the most publicly active) replicate the refusal bypass gap on their own internal steering benchmarks within the next two quarters. If the non-linear advantage holds on models above 70B parameters, that would pressure the field to retire linear-only tooling in production red-teaming pipelines.