Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Muon is an optimizer that builds a momentum update for a weight matrix and then approximately orthogonalizes that update with a short Newton–Schulz iteration before applying it. It is designed for matrix-shaped parameters, not for every tensor in a model. The reported gains come from specific experiments, most notably Moonshot AI and UCLA’s 2025 technical report Muon is Scalable for LLM Training, which found comparable performance to AdamW at roughly 52% of the training FLOPs in its compute-optimal scaling experiments. Those results are a strong reason to test Muon, not a guarantee that it will beat AdamW in your setup.
What Muon actually orthogonalizes
The name is usually expanded as “MomentUm Orthogonalized by Newton-schulz.” The part that often confuses readers is the target of the operation. Muon does not orthogonalize the model’s stored weights. It orthogonalizes the update for a weight matrix: the direction the optimizer wants to move that matrix on the current step.
As an Amazon Associate I earn from qualifying purchases.
Most adaptive optimizers, including AdamW, scale each coordinate of the gradient-based update using per-element statistics. Each entry of the matrix is treated largely on its own. Muon instead looks at the update as a whole matrix. An orthogonal matrix has all singular values equal to one, so orthogonalizing the update roughly equalizes how strongly the step pushes along each of the matrix’s principal directions. That is the geometric idea behind the method: respect the matrix structure of a linear layer rather than adjust each entry in isolation.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
How one Muon step works
At an explanatory level, a Muon update for one matrix parameter follows four stages. The exact configuration depends on the implementation you use.
#1 Best Overall
- A USB accessory that brings machine learning inferencing to existing systems. Works with Raspberry Pi and other Linux systems
- Performs high-speed ML inferencing: the on-board edge TPU Coprocessor is capable of performing 4 trillion operations (tera-operations) per second (tops), using 0.5 watts for each tops (2 tops per watt). For example, it can execute state-of-the-art mobile vision models such as mobilenet V2 AT 400 FPS, in a power efficient manner
- Works with Debian Linux: connects to any debian-based Linux system with an included USB 3.0 Type-C cable
- Supports tensorflow Lite: no need to build models from the ground up. Tensorflow Lite models can be compiled to run on the edge TPE
- Supports automl vision edge: easily build and deploy fast, high-accuracy custom image classification models to your device with automl vision edge
- Accumulate momentum. The optimizer keeps a running momentum of the gradient for the matrix, the same basic role momentum plays in SGD-style methods.
- Form the matrix update. The momentum buffer for that parameter is treated as a two-dimensional matrix. This is why Muon applies to matrix-shaped parameters.
- Approximately orthogonalize with Newton–Schulz. A short iteration of matrix polynomial steps pushes the update toward its orthogonal factor, the matrix you would get by discarding the singular-value magnitudes from a singular value decomposition. The iteration avoids computing a full SVD, which is expensive at LLM layer sizes.
- Scale and apply. The orthogonalized update is scaled by a learning rate and any update-scale adjustment the recipe uses, then applied to the weight, with weight decay handled as part of the recipe.
PyTorch’s stable documentation for torch.optim.Muon exposes the number of Newton–Schulz steps and the polynomial coefficients as configuration choices, and it documents several learning-rate adjustment modes. Those names and defaults are version-sensitive, so confirm them against the PyTorch release you are running before copying any configuration.
Which parameters get Muon
Muon’s update rule is defined for two-dimensional, matrix-shaped parameters, such as the projection matrices in attention and feed-forward blocks. Many other parameters do not fit that definition. Biases and normalization gains are vectors, and the embedding and output layers are often handled separately because of their size and role. Published training recipes generally pair Muon with another optimizer, commonly AdamW, for the parameters Muon does not cover.
Rank #2
- Delivers 25 TFLOPS of AI inference performance using four MemryX MX3 accelerators in a compact M.2 2280 M-key module.
- Connects through PCI Express Gen 3 and above for straightforward integration into embedded and edge systems with an M.2 M-key slot.
- Includes 42 MB of distributed on-chip weight storage to support low-latency, batch-1 pipelined inference.
- Designed for low-power deployment with a 1 W–12 W operating range and host-managed airflow requirements.
- Supports ONNX, TensorFlow, Keras, and TFLite models through the MemryX software stack on Linux and Windows hosts.
This split matters when you read a claim like “Muon replaces AdamW.” In the reported setups, Muon handles the matrix parameters and another optimizer handles the rest. Before you attribute a result to Muon, check which parameters used it, because the result depends on that split.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsWhat the scaling evidence shows
The 52% FLOPs result
The most cited number comes from Moonshot AI and UCLA’s 2025 report. In its compute-optimal scaling-law comparisons with AdamW, the authors report that Muon reaches comparable performance using approximately 52% of the training FLOPs. That is a statement about floating-point operations spent to reach a given quality level in those experiments. It is not a measured wall-clock speedup, and it should not be restated as “Muon is twice as fast.”
What makes it work at scale
The report credits two adjustments for Muon’s behavior at larger model sizes. The first is weight decay. The second is a parameter-wise update-scale adjustment, which scales each parameter’s update so its magnitude is kept comparable to what AdamW-style updates would produce. The stated purpose is to make Muon’s update size behave predictably across layers and model sizes, which reduces how much the learning rate has to be retuned. Treat that as the report’s design rationale for its experiments, not a general law.
The Moonlight model
Moonshot’s Moonlight project repository describes a mixture-of-experts model with 3 billion active and 16 billion total parameters, trained on 5.7 trillion tokens with Muon, and it publishes the implementation and released artifacts. Keep the two parameter counts separate. Active parameters are the ones used per token, while total parameters include all experts. The training details are the project’s own report, so they should be cited to the project rather than presented as an independent benchmark.
Rank #4
Muon compared with AdamW
The fairest comparison looks at the same axes for both optimizers. The table below uses the reported evidence and marks what the sources do not establish.
| Axis | Muon | AdamW |
|---|---|---|
| Update geometry | Matrix-level approximate orthogonalization of the momentum update | Coordinate-wise adaptive scaling of each entry |
| Parameter coverage | Suitable two-dimensional weight matrices; other parameters need another optimizer in typical recipes | Applies to all trainable parameters |
| Reported compute efficiency | Comparable performance at approximately 52% of training FLOPs in Moonshot AI and UCLA’s 2025 compute-optimal experiments | Baseline in those experiments |
| Wall-clock throughput | Not stated as a general result; depends on hardware, implementation, and distributed setup | Not stated as a general result; depends on the same factors |
| Tuning burden | Weight decay, learning-rate adjustment mode, update scale, and Newton–Schulz configuration all matter | Established defaults and a long track record across LLM recipes |
| Distributed cost | Orthogonalization needs the full matrix update, which affects sharding and communication | Elementwise state is straightforward to shard |
Why the distributed implementation decides real speed
A FLOP count does not tell you how fast a run finishes. Orthogonalizing a matrix update requires access to that matrix as a whole. When a large model’s weights are sharded across GPUs, the optimizer must either gather the relevant matrix pieces, compute the orthogonalization in a layout that supports it, or approximate it in a sharded way. Each choice has communication and memory costs.
Best Value
- ✅Powered by 26 Tera-Operations Per Second (TOPS) Hailo-8 AI Processor. 2.5W typical power consumption
- ✅Scalable, enabling simultaneous processing of multi-streams & multi-models
- ✅Enabling real-time, low latency and high-efficiency AI inferencing on the edge devices
- ✅Supports TensorFlow, TensorFlow Lite, ONNX, Keras, Pytorch frameworks
- ✅Supports Linux and Windows. Supports the temperature range of -40°C to 85°C
Moonshot’s distributed implementation addresses this in its own codebase, and PyTorch’s engineering guide Using Muon Optimizer with DeepSpeed covers practical distributed use. Results on one cluster, framework, or sharding strategy do not transfer automatically to another. If you are planning a large run, benchmark end-to-end throughput on your own hardware and parallelism layout, and compare time to a target loss, not only FLOPs.
Practical checklist before you adopt Muon
- Confirm which parameters in your model are two-dimensional matrices eligible for Muon, and which will use another optimizer.
- Pin the framework version and check the current
torch.optim.Muonsignature, default Newton–Schulz settings, and learning-rate adjustment mode. - Carry over weight decay and update-scale choices from a tested recipe rather than reusing AdamW hyperparameters unchanged.
- Measure wall-clock time to a target validation loss on your actual cluster, with your sharding and communication settings.
- Run an AdamW baseline with the same data, model size, and token budget, so any difference you see is attributable to the optimizer.
Is Muon better than AdamW for LLM training?
For large language model pretraining, the evidence supports Muon as a serious alternative to AdamW for matrix-heavy architectures, and the 52% FLOPs result is a credible reason to evaluate it. The evidence does not show that Muon wins across models, data mixes, hardware, or distributed setups. The published comparisons come from particular experiments and implementations, and the study’s own scaling conditions define where the 52% figure applies.
The practical answer is to treat Muon as a candidate that needs a controlled comparison. Use it for the matrix parameters it was designed for, keep a tuned AdamW baseline, and decide based on time to quality on your own system.
Quick Recap
“
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
