Hybrid latent attention cuts looped model cache size by 10.7x

Hybrid Latent Attention separates the cache problem into two tiers rather than compressing uniformly. Recent tokens stay exact (preserving precision where it matters most), while older tokens compress into learned latent codes. This two-tier strategy is distinct from prior KV cache work because it exploits the observation that not all positions in context contribute equally to next-token prediction.

This work sits directly downstream of the fixed-point convergence insights from the October 5th paper on looped models. That research identified decoding as a critical bottleneck and proposed terminal key-value sharing; Hybrid Latent Attention solves the complementary problem of cache size itself, enabling the throughput gains that fixed-point methods assume are possible. Together with LoopCD (October 1st), which extracts value from intermediate loop predictions, these three papers form a coherent efficiency stack for recurrent architectures. The latent encoding approach also echoes the attention relay work from October 4th, which showed that model-agnostic signals (in that case, instruction-following) can transfer via attention patterns. Here, the latent codes learn which older context is structurally important.

If Ouro models uptraining with frozen pretrained weights matches or exceeds the performance of full retraining on the same downstream tasks within the next two months, that confirms the technique generalizes without expensive retuning. If not, the 10.7x cache reduction may be buying throughput at the cost of quality that only full training can recover.