FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibration

The key detail the summary gestures past is what 'attention tuning' actually does here: rather than retraining a draft model from scratch, FlexDraft adjusts attention patterns post-hoc to better align the drafter's output distribution with the verifier, which is what makes the no-retraining claim credible rather than aspirational.

FlexDraft belongs to a cluster of inference efficiency work Modelwire has been tracking closely. The TIDE coverage from the same week addresses a structurally similar problem: how to extract more throughput from large models without paying the full compute cost at every step. Where TIDE exploits temporal stability in expert activation to reduce I/O overhead in diffusion-based MoE models, FlexDraft targets the acceptance rate bottleneck in autoregressive speculative decoding. Both papers are essentially arguing that smarter scheduling and calibration can substitute for brute-force compute scaling. CopT, also from this week, adds another angle: adaptive reasoning that skips unnecessary token generation entirely. Together these three papers suggest the inference optimization field is converging on a shared intuition that waste lives in the coordination layer, not the model weights.

If FlexDraft's acceptance rate gains hold across drafter-verifier pairs with larger capability gaps (say, a 7B drafter paired with a 70B verifier rather than matched-scale pairs), that would confirm the calibration approach generalizes. If gains collapse in that setting, the method may be tuned to narrow distribution gaps and less useful in production deployments where mismatched model families are common.