Cross-layer KV-cache compression cuts memory bottleneck for long-horizon agents

BellowsKV's key insight is that KV representations across layers exhibit shared structure exploitable at the latent level, not just within individual layers. This means a single compressed representation can expand on-demand to serve multiple depths, rather than requiring per-layer compression or cross-layer sharing schemes that still maintain separate caches.

This connects directly to the cache engineering work from October 20th, which tackled KV cache I/O as a dominant cost in long-horizon agent tasks. Where that work decoupled cache control from trajectory modification, BellowsKV attacks the problem upstream by reducing the cache footprint itself. The Token-Energy Law paper from the same week formalized how KV cache updates drive power consumption, giving operators a model to measure whether latent-based compression actually reduces realized energy per token. Together, these three papers form a coherent response to the same production bottleneck: as context and batch size grow, cache memory becomes the limiting factor, and solutions must address both allocation strategy and footprint.

If BellowsKV's latent expansion approach maintains quality parity with full KV caches on reasoning benchmarks (like GPQA or ARC-Challenge) at context lengths beyond 32K tokens, and if inference latency doesn't regress compared to standard KV serving, that signals the technique is production-ready. Watch whether major serving frameworks (vLLM, TensorRT-LLM) integrate this within the next two quarters; absence would suggest the overhead of on-demand expansion outweighs the memory savings in practice.