Inner Momentum for Differentially Private Muon
Averaging per-example Muon gradients over recent model snapshots before clipping bounds distortion and improves differentially private GPT-2 fine-tuning.
Stories per day · 30 days
1 RSS
2 stories, newest first · one card per story
Stories tagged #muon-optimizer
Training a 3.8B LLM to 0.384 CORE for $998 – Hugo Vergnes
Independent developer Hugo Vergnes trained a 3.8B-parameter Llama-style model to 0.384 CORE on 65B tokens for $998 in 43 hours on rented B200s.
4