On the Role of Directionality in Structural Generalization

The more pointed finding here is not the accuracy jump itself but what it implies about benchmark design: SLOG has been testing for directional reasoning all along, yet prior architectures were never built to represent it, meaning the benchmark was effectively measuring an absence rather than a capability.

This connects to a thread running through several recent papers on Modelwire. The 'Understanding Large Language Models' survey from July 1 raised the question of which cognitive phenomena are genuine architectural artifacts versus emergent fluency, and this paper offers a concrete case study in answer: compositional generalization is sensitive to whether the right structural primitive is present, not just whether the model is large enough. That framing also rhymes with the quantization work covered in 'Beyond Activation Alignment,' which showed that calibration choices shape which capabilities survive compression. Both papers push against the assumption that scale alone resolves structural mismatches. The directional CCG result is a tighter, more controlled demonstration of the same principle.

Watch whether the SLOG position-shift gains replicate on COGS or SCAN splits that involve similar left-right asymmetries. If they do not transfer, the CCG typing may be overfitted to SLOG's specific construction rather than capturing a general compositional primitive.