Ternary Mamba: Grouped Quantization-Aware Training of W1.58A16 State Space Models
Researchers have cracked efficient ternary quantization for Mamba-2, compressing the 1.3B model to under 750MB while retaining near-parity performance on zero-shot tasks. The breakthrough hinges on quantization-aware training from pretrained checkpoints rather than from scratch, slashing the token budget by three orders of magnitude and completing the full pipeline in 4 GPU-hours. The work surfaces a previously unknown instability in learnable quantization scales that only emerges during fine-tuning, not initial training. This matters for edge deployment of SSMs, where memory constraints have been a hard ceiling, and signals that the efficiency frontier for linear-time models is still being actively pushed.
Modelwire context
ExplainerThe more consequential finding may not be the compression ratio itself but the newly documented instability in learnable quantization scales during fine-tuning, a failure mode that only surfaces when adapting pretrained weights and would be invisible to anyone training quantized SSMs from scratch.
The memory constraint this paper attacks connects directly to two threads in recent coverage. The 'Embedded Machine Learning for Microcontroller-Class Edge Devices' piece from the same day framed edge deployment as a pipeline discipline problem, not just an algorithmic one, and ternary Mamba fits squarely into that framing: getting a 1.3B SSM under 750MB is only useful if the surrounding deployment stack can actually handle it. Separately, the 'Memory as a Wasting Asset' piece on flash endurance for embodied agents raised a related constraint: it is not just how much memory a model occupies but how write patterns interact with flash longevity. A model that fits in flash is a prerequisite for that cost calculus to even apply. Together, these three papers sketch an emerging infrastructure problem for on-device inference that no single paper fully addresses.
Watch whether the instability in learnable quantization scales reproduces when this approach is applied to larger Mamba-2 variants (3B and above), since the 1.3B result leaves open whether the fix generalizes or was tuned to that checkpoint's specific weight distribution.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsMamba-2 · Slender-Mamba · Bi-Mamba · State Space Models · quantization-aware training
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.