What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers

The paper draws a meaningful distinction between two separate cognitive operations: planning, which can happen in surprisingly shallow networks, and commitment, which requires greater depth to sustain. That asymmetry is the real finding, and it has direct implications for how we evaluate whether a small model is actually 'reasoning' or just pattern-matching early and holding on.

This connects most directly to the 'Generalization in LLM Problem Solving' paper from the same week, which found that models handle spatial transfer well but collapse on longer planning horizons. That paper diagnosed a failure mode from the outside; this one opens the hood and points to specific attention heads as the mechanism. Together they suggest the bottleneck in small-model planning isn't knowledge but architecture depth, specifically the layers needed to lock in a decision without drifting. The looped transformers work ('Stability and Generalization in Looped Transformers') is also adjacent here: if commitment requires depth, then test-time compute scaling via looping may be one way to give shallow models more runway without increasing parameter count.

If follow-up work shows these same commitment-sustaining attention heads are active during the horizon-length failures identified in the shortest-path generalization paper, that would establish a concrete mechanistic link between depth constraints and systematic planning collapse in small models.