Jerry Tworek

Small step for a finger on "post tweet" button. Big step for millions of future AI agents doing useful work for us.

It was an honor to give the keynote at MLSys Covered how AI systems have evolved, why AI is needed to improve them, why results have disappointed, why the future looks amazing, and why I’m working on this at Core Auto Recording should be out soon, in the meantime slides

@babdam 🤣🤣

@MillionInt bitter is taken, would you like the Rancid Lesson™

Low bias high variance methods are usually not very compute efficient but they are scalable High bias low variance methods have higher efficiency but hit limits of scaling quickly One of those sides has had better time historically

@joannejang 🤣

they were so so early

@tszzl @tenobrus I still haven’t used the app. I’ll be there on Alpha Centauri in 2077 and I’ll be deep in those tuis in tmux

@tenobrus friends don’t let friends use the TUI download the app rn

wow that must have taken a lot of courage for codex to admit

Mathematics is a science of transforming objects you’re working with until the answer to the question you’re trying to answer becomes trivial

Best productivity hack I know is organizing your work so that you enjoy it the most

The technology is already largely here, with consistent trend over many years, disrupting consumer and industry workflows. Demonstrating leaps of progress each year. Building largest companies in the world in the blink of an eye. And yet likely less than ten people in the world don’t think of it in too small terms.

It is surprising how many lessons are next to impossible to verbalize. Human mind is surprisingly good with reasoning about uncertain and nonquantifiable qualities. Vibes so far have been very hard to replicate in silicon.

@willdepue Golden age of golden ages ahead

golden age for optimizers right now. every day you see another SoapyShampooGluon ^-1/2 (RMSMatched) drops

@Sheikheddy @_arohan_ Haven’t seen anyone using batchnorm for a long time

@MillionInt @_arohan_ Why layernorm and not, say, batchnorm?

@_arohan_ Layer norm 100%

when Anthropic represents 50.1% of GCP revenue they’re contractually required to rename it Google Claude Platform

@DakshMalik47 @_arohan_ @_arohan_ tweets every time he has a good research idea

4 billion dollars, just to fund @_arohan_ tweetmaxxxing BEST Decision @MillionInt

If the AI models are so smart, why do I feel like I’m losing a few neurons every time I read a longer form content written by AI? We’ve come a long way but we still have long way to go. In terms of clarity of writing we may have regressed from o1/o3 days.

My entire timeline is talking about mangoes this morning and I don’t know why. Good morning!

As there are a lot of questions what I meant here: If there is heuristic for solving the game simple enough to be coded by codex, RL will find it and optimize towards it. But it is clear that such heuristic is not generally intelligent and will not generalise. The bottleneck to RL generalization lies in environments where simple heuristics get you far and it’s easy to get stuck in local optimum.

This is unfortunately mostly a bearish take on RL

@joannejang Love doug but he’s not very smart

@MillionInt doug will hear about this

i do love objectifying my hot chindian husband