Adaptive data structuring cuts LLM agent costs by 28X on reasoning tasks

The paper identifies that the real cost driver isn't reasoning itself but repeated token consumption scanning unstructured documents for scattered evidence. The 28X gain assumes pre-structured data, which sidesteps the harder problem: how to structure documents automatically without human annotation or massive upfront cost.

This connects directly to the semantic chunking work from late August, which tackled a similar bottleneck in RAG pipelines by replacing fixed-size chunking with entity-aware boundaries. Both papers recognize that how data is organized upstream determines whether downstream reasoning (retrieval or agent planning) becomes economically viable. The difference: this token-efficiency work focuses on inference cost per query, while the biomedical chunking paper emphasizes accuracy and interpretability. Together they suggest the field is converging on a pattern: pre-processing and structuring are no longer optional luxuries but prerequisites for production deployment.

If the authors release a follow-up showing that automated structuring (without human annotation) closes the gap to even 10X efficiency gains on held-out document types, that confirms the approach scales beyond curated benchmarks. If the 28X number holds only on FanOutQA-like datasets with clean entity hierarchies but degrades on messier real-world corpora, the practical ceiling is much lower than claimed.