LASER: Low-Rank Activation SVD for Efficient Recursion

The key insight isn't compression itself but the claim that recursion specifically creates this low-rank structure, meaning weight-sharing models may be geometrically different from standard transformers in ways that make them uniquely amenable to activation-space compression without any retraining penalty.

This sits at the intersection of two threads Modelwire has been tracking. The 'Stability and Generalization in Looped Transformers' paper from April 16 established that recursive (looped) architectures have distinct fixed-point dynamics compared to standard transformers, and LASER now suggests those dynamics leave a detectable geometric signature in activations that can be exploited at inference time. Separately, 'K-Token Merging for Large Language Models' from the same day approached inference compression from the sequence dimension rather than the activation dimension, using a learned encoder to reduce token count. LASER's approach is lower-level and requires no additional learned components, which is a meaningful practical difference. Together, these papers reflect a broader push to find compression handles that are native to a model's structure rather than bolted on afterward.

The critical test is whether LASER's compression ratio and accuracy trade-off holds on recursive models with varying loop depths beyond the configurations reported in the paper. If third-party benchmarks on publicly available looped architectures replicate the efficiency gains within the next two quarters, the method has legs; if results are sensitive to loop count or task type, the low-rank assumption may be narrower than claimed.