Muon optimizer boosts agentic RL performance 88% over AdamW
Muon optimizer shows substantial gains for reinforcement learning post-training on language models, outperforming AdamW in sparse-reward agentic tasks. Testing on Qwen2.5-0.5B with ALFWorld benchmarks revealed an 88% success rate improvement under GiGPO when Muon targeted hidden weight matrices, with performance varying significantly by advantage estimator and learning rate. The findings suggest optimizer choice materially affects RL convergence in agent training, challenging assumptions that Muon's value is limited to pre-training phases and opening questions about hyperparameter sensitivity in policy optimization workflows.
Modelwire context
ExplainerThe paper's real finding is not that Muon works, but that optimizer selection during policy optimization is a first-order lever that prior work has largely ignored. The 88% gain is conditional on specific choices (hidden weight targeting, particular advantage estimators), meaning naive application will fail.
This connects directly to the PEARL framework from earlier today, which identified sample inefficiency as RL's core bottleneck in high-dimensional control. Where PEARL adds physics priors to reduce exploration cost, this work shows that algorithmic choices within the optimizer itself can materially shift convergence speed on sparse-reward tasks. Both papers challenge the assumption that RL's scaling problems are primarily about data or architecture, not tuning. The implication is that RL practitioners may be leaving substantial performance on the table by treating optimizer selection as a hyperparameter sweep rather than a structural choice.
If Muon with hidden-layer targeting reproduces the 88% gain on a held-out benchmark (not ALFWorld) within the next six months, it signals the effect is robust. If instead the gains collapse on new environments or require per-task retuning of learning rates and advantage estimators, the finding is narrow and the paper is primarily a cautionary tale about benchmark-specific tuning rather than a general principle.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsMuon · AdamW · Qwen2.5-0.5B-Instruct · ALFWorld · GiGPO · GRPO
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.LG originally reported this story as “When Does Muon Help Agentic Reinforcement Learning?”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.