Symmetry principle unifies neural network learning plateaus and scaling laws

The paper's core contribution is deriving a single quadratic form from permutation invariance that explains both grokking (sudden jumps) and power-law scaling. What the summary glosses over: this works specifically for early-stage learning, not the full training trajectory, and the 'handful of collective variables' still requires empirical validation across real architectures to move from theory to prediction.

This connects directly to the structural limits work from August 13 (reference 2), which argues that ML performance is bounded by data properties rather than algorithmic cleverness alone. Neural Quadratic Forms takes the opposite angle: it proposes that training dynamics themselves have hard mathematical structure independent of the specific loss landscape. Together, these suggest a two-layer constraint model (data limits your ceiling, training geometry determines your path). The SORT paper (reference 6) on sparse equation discovery from noisy data also shares the goal of extracting compact mathematical descriptions from complex empirical behavior, though applied to dynamical systems rather than neural training.

If researchers successfully predict grokking onset (layer count, learning rate, dataset size) on held-out architectures using only the quadratic form parameters within the next 6 months, the theory has moved from explanatory to predictive. If predictions fail on modern large-scale models or require architecture-specific tuning, the early-stage restriction becomes the limiting factor for practical scaling law forecasting.