gabriel teston

Training LLMs across multiple datacenters is hard. 🛑 Synchronization demands often cause massive slowdowns as we scale up. If you're at @NeurIPSConf, come see how we tackle this! Our work, "Scaling Laws for DiLoCo," shows how DiLoCo relax synchronization without compromising model quality, allowing training to scale incredibly well. Come chat with me and @NovaFallen8: 🗓️ Thu, Dec 4 ⏰ 11 AM – 2 PM PST 📍 Exhibit Hall C,D,E, #811 #NeurIPS2025 #LLMs #DistributedTraining #ScalingLaws

It is today! ⏰ 11 AM – 2 PM PST 📍 Exhibit Hall C,D,E, #811

Heading to @NeurIPSConf in San Diego. I’ve got some DiLoCo stickers to give away! 👾 ❤️ Come check out our poster. 🗓️ Thu, Dec 4 ⏰ 11 AM – 2 PM PST 📍 Exhibit Hall C,D,E, #811 #NeurIPS2025

Attending @NeurIPSConf and interested in distributed, modular, and/or open AI? Hadn't seen someone put together a list of poster presentations in this area so took it upon myself to thread out who I'm excited to talk to next week🧵

@yacinelearning 😭

@GabrielTeston I wish I could man but just came back from california

how to attend ai research conferences the yacine™️ way

@yacinelearning Are you going to NeurIPS boss?

hope it helps also for poster sessions try to find the ones doing RL on your favorite game and ask them very precise questions about the game (not their work) to test out their skillset

@eliebakouch @Ar_Douillard should we start calling it “planets” instead of “islands” already?

> Project Suncatcher is exploring how we could one day build scalable ML compute systems in space, harnessing more of the sun’s power It appears that interplanetary diloco wasn’t a joke after all 🚀

We have TPUs in space. I have a DiLoCo implem running on TPUs. Cosmic Distributed Learning when?

@gabriel1 @nikitabier @gabriel @elonmusk Would it be like a UFC, can I challenge you after? (If you win)

ok @nikitabier challenge accepted for the @gabriel username @elonmusk you and me friendly cage match, let me just gain couple pounds. give me 6 months

@yacinelearning @DanAdvantage Living my best life 🙏

@GabrielTeston @DanAdvantage damn man you got the job the girl and the view what a life

ultimate flex thread; now's your chance to brag without being judged. > 5 things you accomplished this week i'll go first: 1) lost ~1 pound, no muscle loss 2) ran 35 miles 3) upped father game, paying extra attention to kids and engaging them 4) @yacinelearning interview 5) $$$$

@DanAdvantage @yacinelearning One of the best. Btw I was here when I got the news about the promotion

@GabrielTeston @yacinelearning dang, successful week! nice work!

@yacinelearning @DanAdvantage Thanks boss

@GabrielTeston @DanAdvantage congrats on the promotion boss!!!

@Ar_Douillard Super deserved!!!

I’ve been promoted to Staff RS. Vain title etc. but feels good to see appreciation for distributed learning in DeepMind ☺️

I would say probably get a full time role as RS/RE, maybe spend some time abroad and connect to more smart people 🤓

@eliebakouch @yacinelearning Nous is looking for a dj for their after party…🤭

@yacinelearning here you go boss, there is some dj set here soundcloud.com/eliemusicc

what are you 5 years goal folks

@Ar_Douillard Huge!

Learned today that a startup is using Streaming DiLoCo to train a distributed AlphaFold-like model. Happy :)

@riseofreh @VictorTaelin Que porra é essa?

@VictorTaelin Minha função era escavar, procurar por timestamps, snapshots, vídeos espelhos, e coisas bizarras que são ocultas pra maioria das pessoas. Você navega por uma curva crítica... Eu por outra...

At this point everyone, even OpenAI, is annoyed at me posting Codex stuff non-stop. You get it. I know you do. But I'm genuinely having so much fun and progress building HVM4 that I have to contain myself not to post every small thing. So many things that were previously just not viable due to time constraints are now within reach. For example, HVM1 needed a major refactor of the GC before we could fix the parallel bug. That would take days. I never had that time. HVM4 often needs major refactors to push things forward too, except, now, I can do that by writing a 30 minute prompt, rather than 2 days of coding. In the last 2 days, I iterated through 5+ completely different approaches to task parallelism on HVM4's runtime. Most attempts went nowhere, but that means I lost minutes, not days, as it would take previously. Writing concurrent code is HARD. So many subtle errors. Should I put a fence here? Should this CAS use seq_rel? Should this field exist? Add a subtle bug and there goes an evening debugging... Now, I can just draft a half baked prompt that explains what is on my mind in a way that at least makes some sense, and the AI will fill the holes implement my idea as I browse X. 99% of the times, it just works. Sure, it is most likely just recalling details it learned from human literature. "Okay to write a MPMC, I need this atomic here, then this fence here." But applying it to my specific setup in 10 minutes is nothing but miraculous. I'm just thinking about the last 2 days and there is absolutely no way I'd be able to have tried so many different things in so little time. Now HVM4 has a parallel runtime that works, scales well with cores, all tests pass, near 0 error rate. 2 days ago I had none of that. HVM3 spent a whole year without one!

@eliebakouch 😮

Open source release of smol merch coming soon at colm 🤏

Want to learn how to train models across the world, with 400x less bits exchanged and a huge latency tolerance? 🌎 I’ll be presenting our work on how to efficiently scale distributed training at @COLM_conf. 🗓️ TODAY: Tuesday, 11:00 - 13:00 📍 Room 710 #COLM2025