Tool graph memory fails to justify computational cost in LLM agents

The paper doesn't just show memory underperforms; it suggests the performance lift from any memory mechanism may come from extra compute cycles rather than learned structure. This is a methodological red flag: if you're comparing a memory-augmented system against a non-augmented baseline, you need to control for total inference budget, not just model calls. The authors appear to have done this, but the implication (that random guidance works nearly as well) deserves scrutiny on whether the 'learned' memory was actually learning anything or just adding noise.

This connects directly to two recent findings on agent memory. Microsoft's Fork-and-Flush work (October 12) showed that context management, not accumulation, drives agent reasoning quality, and the 'Scaling Discovery Requires Forgetting' paper (October 14) identified that unbounded memory locks agents into local optima. Both suggest memory overhead without strategic pruning is a liability. This tool graph study extends that pattern into tool-use specifically, but with a sharper claim: the memory isn't just unhelpful, it's indistinguishable from random noise at twice the cost.

If the same nine models show tool graph memory gains when evaluated on a held-out tool set (unseen during memory construction), that would vindicate the mechanism. If gains persist only on in-distribution tool sequences, the random guidance hypothesis holds. The critical test: does performance scale with memory size, or does it plateau immediately? Flat scaling would confirm the finding; linear scaling would suggest the paper's baseline comparison was flawed.