Bit unsettling how there's so many LR conventions for using Muon/Normuon and we don't definitively know the benefits/drawbacks of each one yet

@Creative_Math_ Bonus: DeepSeek's Nesterov Muon also does "RMS-matching", but with a factor of 0.18 instead of 0.2 😂 x.com/EIFY/status/20…

I reject RMS matching as a whole, it changes the geometry and leads to attention logit instability even in small models (which means sth is def wrong there). I’m not convinced apriori that the update RMS for 2 diff optimizers should be the same, esp not when it’s muon (which has a specific geometry)

@Creative_Math_ I agree in principle: I don't use "RMS matching" for my own experiments. Though in practice it probably served as a quick & dirty bridge for Kimi / GLM / DeepSeek to transition from AdamW to Muon while reusing the tuned LR.