VestigeKV uses NoPE model internals to guide KV cache eviction
VestigeKV solves a critical bottleneck in long-context inference for linear attention models by repurposing an architectural artifact into a cache eviction signal. Rather than relying on observed attention patterns (which fail when tokens haven't been queried yet), the method reads a 64-dimensional branch that NoPE training already embeds as a salience proxy, partitioning the KV cache into attended and archived tiers without training, quantization, or kernel changes. On Kimi Linear, retrieval accuracy remains perfect while reducing memory pressure, suggesting that vestigial model components can encode useful metadata for inference optimization.
Modelwire context
ExplainerThe key insight is that NoPE training already encodes salience information in a 64-dimensional branch that was never meant for inference. VestigeKV repurposes this artifact as a free eviction signal without retraining, quantization, or custom kernels, which means the optimization works on existing deployed models.
This connects directly to the LatentPress work from early September, which also bypassed traditional reconstruction to compress context more efficiently. Where LatentPress encodes long contexts into continuous embeddings at the decoder layer, VestigeKV solves the complementary problem: once you have a KV cache, how do you decide what to keep? Both papers treat inference compression as a latent-space problem rather than a text-level one. The broader pattern across recent work (MemoryWalker, LatentPress, and now VestigeKV) is that production systems are moving away from explicit attention patterns or summaries toward implicit signals already present in model internals.
If Kimi Linear's retrieval accuracy stays perfect on the LongBench-Retrieve benchmark when cache size drops below 50% of baseline, that confirms the 64-dimensional branch is genuinely encoding task-relevant salience and not just correlating with it. If other linear attention models (H2O, SnapKV) adopt this method without retraining within the next two quarters, that signals the technique is portable across architectures.
Coverage we drew on
- LatentPress: Context Compression Beyond Text and Vision · arXiv cs.LG
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsVestigeKV · Kimi Linear · NoPE · H2O · SnapKV
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “VestigeKV: The NoPE-MLA KV Cache Carries Its Own Eviction Signal in a Vestigial Branch”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.