The Inference Shift

The piece leaves one critical question unaddressed: which cloud providers are already positioned for this shift versus which are still building capacity around interactive-latency assumptions, and whether that gap is months or years wide.

The throughput-over-speed argument lands differently when read alongside WIRED's recent piece on CUDA's role in Nvidia's competitive position. If agentic workloads deprioritize latency, the pressure on Nvidia's H100-class hardware softens somewhat, but CUDA's software lock-in becomes even more durable because batch-oriented, cost-sensitive buyers still need the tooling ecosystem, not just raw silicon. Separately, the Hollywood labor piece from the same day is a useful reminder that the human infrastructure behind AI training sits upstream of the inference layer Thompson is describing. The workers annotating data today are feeding the models that will eventually run as the autonomous agents reshaping datacenter economics tomorrow. These are different parts of the stack, but the same structural story: costs and value are being redistributed in ways that aren't yet visible in headline numbers.

Watch whether any major hyperscaler (AWS, Google Cloud, or Azure) announces a pricing tier or hardware configuration explicitly targeting agentic batch workloads within the next two quarters. A concrete product move would confirm that Thompson's infrastructure thesis has crossed from analysis into operator roadmaps.