Muon Optimizer: Mathematical Intuition, CUDA Kernels, and MNIST Benchmarks
A ground-up Muon optimizer implementation exploring Newton-Schulz orthogonalization, CUDA kernels, and MNIST benchmarks against AdamW.
A ground-up Muon optimizer implementation exploring Newton-Schulz orthogonalization, CUDA kernels, and MNIST benchmarks against AdamW.