Neural Garbage Collection: Learning to Forget while Learning to Reason

The key distinction here is the word 'learned': most existing KV cache pruning schemes use fixed heuristics like attention score thresholds or recency windows, whereas this work trains the model itself to decide what to discard, making forgetting a first-class part of the reasoning process rather than a post-hoc optimization.

The closest thread in recent coverage is IG-Search (arXiv cs.CL, April 16), which also tackles the problem of making intermediate reasoning steps more efficient, in that case by rewarding only search queries that genuinely improve answer confidence. Both papers are working on the same underlying constraint: chain-of-thought reasoning is expensive per step, and the cost compounds. K-Token Merging from the same week approaches a related pressure point from the compression side, collapsing token sequences in latent space before they ever enter the model's full attention stack. Neural Garbage Collection is the runtime complement to that static compression idea: rather than compressing inputs upfront, it prunes the working memory mid-reasoning. Together these papers sketch a broader effort to make long-horizon reasoning tractable without simply scaling hardware.

The credibility test is whether the learned gating generalizes across reasoning benchmarks beyond the paper's own evaluations. If an independent group reproduces the memory savings on MATH or ARC-AGI without accuracy regression, the end-to-end training claim holds; if accuracy degrades on harder problems, the model is likely learning to forget things that look redundant but aren't.