Research·arXiv cs.CL·15h ago

KV-cache injection outperforms fine-tuning for knowledge adaptation

A new study systematically compares three architectural approaches for injecting knowledge into LLMs: context windows, parameter fine-tuning, and latent representation caching. The research finds that KV-cache based methods (Cartridges) achieve superior accuracy across most storage budgets, outperforming fine-tuning by 10 percentage points in oracle conditions. This work matters because it clarifies the efficiency frontier for knowledge adaptation, helping practitioners choose between expensive parameter updates, context bloat, and emerging representation-injection techniques. The findings suggest that how information is stored in a model fundamentally shapes both performance and deployment cost, reshaping decisions around RAG, fine-tuning, and inference optimization.

Modelwire context

Analyst take

The paper doesn't just rank methods; it quantifies the storage-accuracy frontier across budget constraints. The 10-point oracle gap between KV-cache and fine-tuning is meaningful, but the real finding is that representation caching scales more gracefully than either context expansion or parameter updates as budgets tighten.

This connects directly to two concurrent threads in your archive. First, the pruning degradation study (September 15) mapped which model components break under compression; this paper shows where knowledge should live to survive that compression. Second, the calibration work on distillation (same day) addressed inheriting teacher errors during compression. Together, these three papers sketch an emerging consensus: the physical location of knowledge in a model (parameters, cache, activations) determines not just accuracy but robustness to downstream optimization. The representation-injection focus here echoes the belief-state geometry paper from the same day, which showed LLMs already encode structured representations; this work suggests we should exploit that structure for storage efficiency rather than fight it.

If Cartridges maintain their accuracy advantage when tested on out-of-distribution domains (domains not seen during cache construction), the method is genuinely robust. If performance collapses on distribution shift while fine-tuning holds steady, the KV-cache gains are brittle and the practical trade-off reverses. Watch for follow-up work testing this within 6 months.

Coverage we drew on

What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity · arXiv cs.CL

This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.

MentionsCartridges · Compaction

Read full story at arXiv cs.CL →(arxiv.org)

Modelwire Editorial

This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.

Our mission How we write

Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Where Should a Document Live: Context, Representations, or Parameters?”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.