Geometric model of language model failure modes predicts brittleness across architectures

The paper's core contribution isn't just mapping failure modes but showing they're directional: identical semantic content can trigger collapse or stability depending on which dimensions of the input space shift. This suggests robustness interventions can be surgical rather than wholesale.

This work sits squarely in the interpretability cluster from the past week. Like the 'Present but Hard to Read' study (Oct 20), it challenges assumptions about what's actually happening inside models versus what surface behavior suggests. But where that paper showed information can hide in representational noise, this one goes further: it proves failure isn't a binary property of a task but a property of the trajectory through input space. It also connects to the 'Aphasia' ablation work (Oct 20), which found architectural differences in degradation patterns. Here we see those patterns aren't random but decomposable into three interpretable components, offering a more actionable framework for the kind of targeted hardening that ablation studies hint at.

If the authors successfully transfer their compact predictive model to Claude 3.5 or Llama 3.1 (architectures not in the training set) with >70% accuracy on failure prediction, that validates the geometric framework as architecture-agnostic. Partial transfer is claimed but not quantified; full numbers on held-out architectures by Q1 2027 would confirm whether this is a general principle or Qwen/Gemma specific.