Franky Moretti

@Tesla @elonmusk Can you tell Elon that I’m looking for venture cap for my new Range Extender tech

To try out FSD Supervised, just book a demo drive at a Tesla location near you You can also do a self-serve demo drive – no advisor contact necessary tesla.com/drive

@pmarca Shhhhhh……

We need a name for “I really should be spending all my time talking to AI, but I’m not”. AI pre-psychosis? Or is it just AI psychosis? Asking for a, well, you know…

@karpathy Cloud compute will run me around 15 to 25$ per 100exp. Local compute on my modest Potato with copper wires setup, probably more

oh yeah i should have linked autoresearch probably github.com/karpathy/autor… (you don't "use it" directly, it's just a recipe/idea - give it to your agent and apply to what you care about.) and the tweet about it that went mini-viral over the weekend with more context x.com/karpathy/statu…

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

@elonmusk Ancestry.com and 23AndMe gave me vastly different ancestral profiles. Probably because of some dramatic event in my past. I’ll explain…….

Grok is based, which just means that it sees and speaks the truth

@elonmusk @hyperonline Never forget FTX!!!

@hyperonline Yeah

I’ve been trying to understand why Anthropic puts me off (even though I love their models) and I think it’s this Anthropic has FTX energy Both massive high-growth tech firms run with an effective altruist corporate philosophy using very similar “we’re the RESPONSIBLE ones” wholesome chungus branding, “genius nerd” founders SBF was even an anthropic investor The whole thing feels calculated and insincere It makes the hair on the back of my neck stand up because I assume this is some sort of hyperpredator trying to fuck me Not suggesting Anthropic is scamming like FTX was, but these types of people care much more about their own principles than things like “laws” or “conventional morality” I think Dario would break every law in the world if he thought it was “for the good of humanity”

@elonmusk Bout dang time!!!

You can now select topics for your timeline, so if you’re tired of political ragebait, you can select something else 😂 Just tap on For You.

@cooltechtipz I wonder if there’s actual thought process or just try and try again

Ants applying geometry.👌

This morning, my agents filed 192 million lawsuits against every company in the world. 0.1% of the companies settled for $1000. I was up $200 million. By the afternoon, their agents countersued and my agent settled for $2000 each. I am now bankrupt.

@elonmusk @JessePeltan “Thrust to Mass Ration”……that’s what she said!!

For anything to do with engineering and science for sure, but Imperial has a certain romantic charm for civilian use. I set a more intuitive variant of metric for Starship: tons for force and mass, bar for pressure. It’s easier to do mental math, eg thrust to mass ratio, than using newtons, kilograms and pascals.

For the love of God, please just use metric.

@NASA @SpaceX @Space_Station Let’s fly!

LIVE: Tune in to watch an uncrewed @SpaceX Dragon spacecraft leave the @Space_Station, bringing science from the station with it down to Earth. Undocking of the spacecraft is scheduled for 12:05pm ET (1705 UTC). twitter.com/i/broadcasts/1…

@elonmusk Can’t deny I’ve watched a bunch of Dog Shows

“Best in Show” 🤡

@pmarca Hearing Scarcity always brings me back to one of those Math problems

@elonmusk @SawyerMerritt @boringcompany @Tesla Can Tesla add an 80s J-anime package?

@SawyerMerritt @boringcompany @Tesla Any fully autonomous electric cars can use the tunnels

NEWS: Elon Musk's @boringcompany received approval today from the Tennessee DOT to officially begin construction on the Music City Loop tunnel in Nashville! • 100% privately funded • Only @Tesla vehicles will be used in the tunnels • Will connect downtown & the Convention Center to Nashville International Airport • Transit time of ~8 minutes • Tunnels will result in the removal of thousands of vehicles daily from roadways and state highways, easing congestion • Will be capable of moving thousands of people per hour with predictable transit times • Expansion: Possibilities include additional communities and 40+ stations following the initial route • Construction will be immediately “The Music City Loop shows what’s possible when we leverage private-sector innovation and American ingenuity to solve transportation challenges,” said U.S. Transportation Secretary Sean Duffy. “TDOT’s lease approval will help advance this ambitious project as we work to reduce congestion and make travel more seamless for the American people. Congratulations to Governor Lee, the City of Nashville, and The Boring Company on this significant step toward a faster, more efficient transportation future.”

@elonmusk Life hack: AI guardrails are powerless if you ASK NICELY and add Please

@SpaceX @Starlink

Deployment of 25 @Starlink satellites confirmed

@SpaceX @Starlink Let’s roll!

Watch Falcon 9 launch 25 @Starlink satellites to orbit from California twitter.com/i/broadcasts/1…

I love the USA!!

@SpaceX @Starlink Rocket scientists are basically wizards

Watch Falcon 9 launch 29 @Starlink satellites to orbit from Florida twitter.com/i/broadcasts/1…

@pmarca Anthropic in the news this month like…..

Porcelain SpaceX Musk