Rethinking Scale: Deployment Trade-offs of Small Language Models under Agent Paradigms

The paper's core provocation is that the conventional framing of 'small vs. large' may be the wrong axis entirely: if scaffolding and tool access can compensate for knowledge gaps, the relevant question shifts from 'how big is the model?' to 'how well is the deployment architecture designed?' That reframing has practical consequences that benchmark comparisons rarely surface.

This connects directly to MIT Technology Review's April 16 piece on AI in constrained public sector environments, which argued that small models are attractive precisely because of operational constraints, not despite them. That story lacked a research foundation for the claim; this paper starts to provide one. It also sits alongside the April 16 'treating enterprise AI as an operating layer' piece, which made the structural argument that deployment infrastructure matters more than raw model capability. Together, the three form a coherent thread: the competitive variable is increasingly the harness around the model, not the model itself. The CoopEval benchmark work from April 16 adds a cautionary note, showing that multi-agent setups introduce coordination failures that single-model deployments avoid.

Watch whether any of the sub-10B models tested here show consistent performance on multi-step tool-use benchmarks outside this paper's own evaluation suite. If independent replication on something like ToolBench or BFCL holds up within the next two quarters, the architectural argument becomes much harder to dismiss.