Google speeds up Gemma 4 threefold with multi-token prediction

The detail worth noting is that multi-token prediction drafting is being applied to an open-weight model at deployment scale, not just benchmarked in a research setting. That distinction matters because it means third-party operators running Gemma 4 on their own infrastructure inherit the speedup without any model changes on their end.

This connects directly to the inference optimization thread running through recent Modelwire coverage. The 'Make Your LVLM KV Cache More Lightweight' piece from May 1st identified GPU memory as the binding constraint in production deployments; Google's move here attacks a different bottleneck, autoregressive latency, but the underlying pressure is the same: inference cost is now a primary competitive variable, not an afterthought. Together these stories sketch a pattern where the optimization layer is becoming as contested as the model layer itself. Open-weight models in particular face a structural disadvantage against proprietary APIs on latency, so techniques that close that gap have outsized strategic value for the open-weight segment.

Watch whether Meta follows with a comparable inference optimization announcement for Llama 4 within the next two quarters. If they do, it confirms that speculative decoding and related techniques are becoming baseline expectations for open-weight releases, not differentiators.