Remote KV cache slower than recomputation in GLM-5.3 production test

The finding isn't just that remote KV caches underperform; it's that this failure mode emerges specifically under context-parallel prefill workloads at production scale. The bottleneck isn't inherent to disaggregation itself but to the synchronization overhead when multiple requests compete for cache hits simultaneously, a scenario most benchmarks don't stress.

This directly contradicts the assumption underlying recent work on hierarchical KV storage. Adaptive DHiSparse (early October) proposed dynamic shifting between GPU and host memory to manage capacity constraints, but assumes network retrieval is viable when needed. The Alibaba finding suggests that assumption breaks under concurrent load. Meanwhile, the CacheFuse and SwaT papers from mid-October explore direct KV sharing between models to avoid serialization costs, but both sidestep the latency question when cache sources are remote. This story forces those architectures to confront a harder constraint: even well-optimized remote access may be slower than recomputation, fundamentally limiting how much disaggregation helps.

If Alibaba or other vendors publish follow-up work showing that local-only KV strategies (no remote tier) outperform disaggregated approaches on the same GLM-5.3 workload at similar hardware cost, that confirms this is a structural problem, not a tuning issue. Alternatively, if network-attached cache systems (Valkey, Redis) ship latency-optimized variants targeting sub-millisecond hits within 60 days, watch whether they reclaim the 2.8x gap on the same benchmark.