cider

linear transformations stretch euclidean space ReLU folds euclidean space neural networks are just repeated origami on high-dimensional laffy taffy

@norpadon (3) is just reservoir sampling right

Some fun ML interview problems

@LotusAbhi @konstmish this has been empirically known for awhile, not sure what the seminal reference is. central flows explains why constant LR prevents sharpness explosion

@jeffreycider @konstmish Do you know of a reference for this? Might explain some behavior of some work I've been doing recently.

A nice and easy to read paper on what the existing Muon literature is missing.

@konstmish in fact, muon was designed as the steepest descent optimizer in a constant sharpness regime (eqn 7 in bernstein's modular duality paper)

@konstmish yes, my criticism is not that it's an apples-to-oranges comparison. it's that all useful NN optimization occurs in the constant-sharpness regime, so a line-search regime is irrelevant

@creatine_cycle @REK god made men but mecha teleoperation made them equal

>show up to robot fight >opponent is 13 >lose

@tenderizzation @distributionat tasty but may shorten lifespan

@distributionat is daeho good if you don't want gimmick instagram food, asking for a friend

My SF food take is that I think San Ho Won isn't very good. Nothing wrong with the food, it simply isn't very good. The banchan is fine, the meat is fine, only the service is one star. Han Il Kwan is good, Hwa Mi Won is good, Daeho is good. I don't understand San Ho Won.

@Impish_Bunny enjoy totk

just finished breath of the wild after putting it off for several weeks :(

@Nexuist @twocents how does a single USD put me in the top quartile

@IgorMezic @Yanlecunn wrong account, it's @ylecun

@tenobrus tokenization in the brain (or any sort of discretization) is really hard to imagine. like whatever neuron performs the discrete selection (like a softmax operation) is going to have to be physically wired to every language-related neuron

do you think humans have something very like autoregressive token completion as one of (altho maybe not only) the core primitives in our language or world modeling?

@yifan_zhang_ the Mu in Muon stands for momentum. the precise claim is that if you strip accumulation out of both Muon and Shampoo, they both give identical steepest descent updates

True, Muon is Shampoo without Momentum.

@skooookum isn't the mta run by new york state

If there was a NYC mayoral candidate whose *only* campaign promise was to make the subways better and increase the construction of new subway no one would be able to touch them.

@bryancsk me and who

Which meme do you think he shared with President Xi

@khoomeik shifting mass between exponential cost coefficients

“when a field is getting started, it’s easy to confuse the essence of what you’re doing with the tools you’re using” what is the essence of LLM Studies? could LLMs just be one instantiation of a broader abstraction? please don’t answer “ml theory”

@SeunghyunSEO7 only in the 124M track. og muon is still better on the 350M track.

just noticed modded-nanogpt adopt 'NorMuon' as default (?). it looks like `AdaMuon`. i personally didnt buy this idea because i thought Muon is enough and dont want to introduce optim state for 2nd moment again like adam... hmm arxiv.org/abs/2510.05491 arxiv.org/abs/2507.11005…

@TsarinaBomba chrysanthemum

@tarashakhurana @CMU_Robotics @RamananDeva @shubhtuls @KaterinaFragiad @cvondrick @GuibasLeonidas welcome!

Life update: I recently defended my PhD at @CMU_Robotics where I was advised by @RamananDeva! Last few years were so much fun. Incredibly grateful to everyone in Smith and especially to my committee @shubhtuls @KaterinaFragiad @cvondrick and @GuibasLeonidas. I am now at @Tesla_Optimus working on cool perception problems for humanoids!

@aspergtame > compute is an inference hyperparameter banger

@tkanarsky we will climb

I'm trash at everything I do.