ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training
Reasoning models face a deployment bottleneck: full-precision inference is expensive, yet aggressive quantization to 4-bit formats degrades chain-of-thought accuracy. ReQAT addresses this by identifying that quantization errors concentrate on low-entropy tokens (digits, operators) where noise cascades through multi-step reasoning. The framework uses trace-aligned quantization-aware training to preserve symbolic precision while compressing weights, activations, and KV caches to FP4. This work matters because it unlocks efficient deployment of long-reasoning models without sacrificing correctness, directly impacting the feasibility of on-device or cost-constrained inference for problem-solving tasks.
Modelwire context
ExplainerThe key insight buried in the methodology is that quantization damage is not uniformly distributed across tokens. Digits and operators, precisely the tokens that carry logical weight in chain-of-thought, are low-entropy and therefore most sensitive to rounding noise, meaning a generic compression scheme will systematically corrupt the steps that matter most.
This sits alongside a broader wave of inference-time efficiency work appearing in the archive this week. The MoE pruning paper ('How to Score Experts for One-Shot MoE Expert Pruning') addresses a structurally similar problem: models that are theoretically deployable but practically bottlenecked by memory and compute at inference time. Both papers are pushing toward the same operational goal from different angles, one through weight sparsity, the other through numerical precision. Neither alone solves deployment for the largest reasoning models, and the interaction between expert pruning and quantization in combined pipelines remains an open question the current literature does not address.
Watch whether ReQAT's FP4 accuracy claims hold on MATH-500 and GPQA Diamond under third-party replication, particularly on problems requiring more than 16 reasoning steps, where quantization error has the most opportunity to compound.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsReQAT · Large Reasoning Models · FP4 · Trace-Aligned QAT
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.