
DMuon: Efficient Distributed Muon Training with Near-Adam Overhead
Muon, a matrix-orthogonalization optimizer, offers superior convergence for large-scale models but has faced a critical deployment barrier: vanilla implementations cost over 2x the compute of standard forward/backward passes due to expensive Newton-Schulz iterations. DMuon resolves this by redesigning distributed training infrastructure to match matrix-level optimization, bringing overhead down to near-Adam levels. This work matters because it removes a practical blocker preventing adoption of theoretically superior optimizers in production, potentially reshaping how teams scale training across heterogeneous hardware as model sizes continue climbing.62




























