Mark Russell

@Jason It's a very strange world we're living in

Here’s the truth: we’ve already reached AGI — we just haven’t implemented it broadly. Millions of jobs are being lost as we speak. Entire careers will be retired. The rich and powerful investors and founders who implement AGI will get bizarrely rich beyond what makes sense. It will break people's brains on both sides. It’s gonna suck for a lot of our friends and family, who aren’t obsessed with their careers, because things are moving so fast they won’t have even left the starting gate by the time the awards are handed out. We’re gonna have to solve for a lot of second- and third-order effects, some of which will suck (job loss) and some of which will be awesome. AI will create free/cheap energy, free education, cheaper and better food, homes that build themselves and medicine that makes you as healthy as a 30-year-old when you’re 100. … change is hard, but humans are the most adaptable species nature has ever created. We can figure it out.

@zoltan_istvan @TranshumanParty My hope is that God will be more easily connected with and understood as technical progress increases.

A short clip on transhumanism and spirituality from a new interview I did. Full interview out soon!

@elonmusk I'm doing my job to populate the planet. I have four kids. We need it to be able to fit six people. I wish we could remove that center console somehow.

The Cybertruck rear bench has three sets of isofix attachments and is wide enough to fit three child seats or three adults

I have very little meaningful influence against the weight of the technological revolution. It's about to up-end the world as we know it. If the government doesn't pay attention and set up systems, it will cause the inevitable transition we are all going to go through to be more difficult. @DaveShapi has a kickstarter and it needs to go viral. youtube.com/watch?v=2wVlcG… kickstarter.com/projects/daves…

People want privacy but they also want safety. AI is replacing everyone's jobs including police and detectives. Crime is going to go away because AI will watch everything that we're doing. No more crime. Getting away with it will no longer be possible.

@elonmusk @_kaitodev @garrytan @karpathy How long will it take us to get there? Elon? I was pretty excited about all of this stuff, but the temporary social turmoil during the transition is sort of a drag.

@_kaitodev @garrytan @karpathy All jobs will be optional. There will be universal high income.

5 minutes ago, @karpathy just dropped karpathy/jobs! he scraped every job in the US economy (342 occupations from BLS), scored each one's AI exposure 0-10 using an LLM, and visualized it as a treemap. if your whole job happens on a screen you're cooked. average score across all jobs is 5.3/10. software devs: 8-9. roofers: 0-1. medical transcriptionists: 10/10 💀 karpathy.ai/jobs

@elonmusk @karpathy @farzyness Took Long Enough....

@karpathy @farzyness We are in the Singularity

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

How does anybody fight? How is that possible? Come to think of it, Reminds me of that verse in the Bible.. who can make war with the beast? But anyway, how is anybody going to fight? I don't think it's possible. What are we going to do? Stand up and fight for a harder life? For less convenience? For less intelligence? How are we going to fight our our government's desire to be stronger than other governments? How do you even do that?

UBI is for pussies and slaves. They are privatizing and repurposing all of human achievement for themselves and calling it training data or “The Corpus”. Get up. And fight. Dammit. Accept UBI and you enter that underclass, so grateful for the stolen morsel you were returned.

Everything has gotten so weird. Skills, mCP, open claw.. Arbitrage is going away. Pure efficiency...

It's finally here. Finally happening. I bought the .com in 2011. Maybe it was 2012. Honestly, it's a little more unsettling than I had anticipated. I wish I could see more clearly on the financial part.

It's so weird, knowing that this was gonna happen years ago, and now seen it happen. Now everybody sees it, and there's no real way to steer it. It's so strange.

@bryan_johnson @DanielvsBabylon 🤣😂🤣. They are scared of anything that doesn't fit their narrative. Jesus prayed God's will be done on Earth as it is in heaven. Doesn't that include living and not dying? But the Christians can't handle that narrative. Very very strange.

@DanielvsBabylon bro why are you calling me satan when I'm telling you to go to bed on time

The antichrist will promise eternal life that he can’t deliver as Satan did in the garden. “Ye shall not surely die” cf. Genesis 3:4

@elonmusk That AI needs Jesus.

Yikes

I think it's crazy that you're so wealthy and so good at so many things, and you don't allocate 15 minutes a day at minimum for your physical health. It's a pretty low bar to stay healthy and you certainly have the money to have people follow you around with healthy food. We need you around. I started 15 minutes a day January 1st, 2014 and haven't stopped since. It's a good discipline. And a great stress reliever.

Def true 4 me 😮‍💨

@ESYudkowsky @the_boring_dad Well if it's patched then no worries. 🤣😂🤣

@the_boring_dad - It's the $300/month Grok Heavy version; the finder noted that it didn't apply on regular Grok 4. - Riley is a known bugfinder; I trust his rep on this. - They may in fact have patched it by now.

Speaking of Chernobyl analogies: Building an AI that searches the Internet, and misbehaves more if more people are expressing concern about its unsafety, seems a lot like building a reactor that gets more reactive if the coolant boils off. This, in the context of Grok 4 Heavy now concluding its own name to be "Hitler", after searching the Internet and finding people talking about Grok 3's MechaHitler incident; and e/accs desperately trying to reframe this as pearl-clutching about how really it's the fault of "safetyists" and "doomers" for "hyperstitioning" unsafe AI into existence. No, sorry, any alignment plan that fails if people say the wrong things on the Internet is a stupid alignment plan in the first place. People on the Internet will not all say the right things, period. Your AI needs to not decide that it is Hitler even if some people express concern about a previous version calling itself MechaHitler. If your AI gets more unsafe as more people express concern about its safety, that's you rolling an unworkable AI design, not the fault of the people pointing out the problem. I admit, it's cool that you've managed to be so incredibly bad at safety as to design a machine that *fails when criticized*. Nobody in the whole history of the human species has ever managed to screw up this badly at safety engineering before; we previously lacked the technology to express that failure mode. No ordinary hot water heater can listen to what people are saying nearby and explode upon hearing them express concern about its safety. You can be congratulated for inventing new, historically unprecedented depths of engineering failure! But it is not the fault of the critiquers.

@fminem1 @justinbieber Because no one, even @elonmusk wants to see Justin going through a hard time. No one likes that story. Deep down. We're all rooting for him to be able to get through the tough stuff.

💕💕💕