Fox 🦊

Finally found this classic 😭

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

Ads are coming to AI. But not to Claude. Keep thinking.

We took @openclaw and brought it to @Cloudflare Workers! With the Sandbox SDK, it's secure by default. Check out the blog post to learn how it all fits together... Or skip straight to deploying 😎⬇️ github.com/cloudflare/mol…

> 1 trillion parameters > cheaper than opus 4.5 > is open source > beats Claude opus 4.5

@tweetciiiim Fake, there was a collision just last month — bbc.com/news/articles/…

AI based monitoring on railway tracks in India has led to zero elephant deaths over the past two years.

@KreatelyMedia Nonsense, literally happened hast month — bbc.com/news/articles/…

AI based monitoring on rail tracks in India Zero elephant deaths in 2 years 🇮🇳

How CNN frames 6% inflation under Joe Biden vs. 2.7% inflation under Donald Trump.

claude --dangerously-skip-permissions

🔥👀🇺🇸 THE LEGEND OF “MAGA TYSON” WAS BORN TODAY. If you put a megaphone with a BLARING SIREN in someone’s ear that w/ the intent to permanently deafen them…that is CONSIDERED ASSAULT & you can defend yourself. He was tossing out HUMPTY DUMPTIES to every single one of them that ATTACKED HIM.

Simply a fantastic shot

@rishibagree @Tamal0401 On what basis are you accusing the company of being involved in coal scam?

@Tamal0401 Why are TMC MPs defending a private company involved in the coal scam ???

I bet you haven’t heard this before “#BJP is ‘beguner bachha’ (son of brinjal)” Says Mahua Moitra Clearly Mohun 🍆 East 🍆 effect

@ODonnell4NH @elonmusk Why do idiots try to block the road and stop law-enforcement? 🤷‍♂️

@elonmusk Why does he keep walking in front of cars?

The same ICE agent was almost killed by car during an arrest six months ago …

@ScoopySnax @elonmusk Don’t block the road and try to stop law-enforcement? 🤷‍♂️

@elonmusk What does that have to do with the woman he murdered? nothing.

@sarkarstix @elonmusk Don’t block the road and try to stop law-enforcement? 🤷‍♂️

@elonmusk That doesn’t give him a license to kill a woman who was clearly trying to save herself and not harm anyone

@PrepperCanadian @elonmusk Don’t block the road and try to stop law-enforcement? 🤷‍♂️

@elonmusk Then dont stand in front of cars like a retard

Just gonna hold the throttle till I see God or the checkered flag

Hilarious 😂

Shekhar gupta held back nothing 💀💀💀

Original "Raghupati Raghav Raja Ram". Note : This is not communal. This is original.