Peter Schafhalter

I’m incredibly grateful for my collaborators at @GoogleDeepMind: Shun Liao, @zhouyanqi30, Chih-Kuan Yeh, Arun Kandoor, and James Laudon. Also, thank you to @ICGog and the Pathways Team for their support and feedback.

We exploit parallelism in MoDE’s architecture to apply flexible sharding configurations that place experts weights on different devices from the pre-trained model’s weights. Such configurations can reduce communication overheads and improve training speeds by up to 38%.

Introducing Modular Domain Experts (MoDE), a new multi-domain adaptation technique for LLMs. MoDE independently trains experts on different domains and composes them to boost LLM performance on complex, multi-domain tasks. Paper: arxiv.org/abs/2410.10181

@vsreekanti NotebookLM is awesome

Feels like the discussion around long context has disappeared recently — are there any big applications of it that are popular right now?

@samkumar_cs @TheOfficialACM @acm_ccs Congrats Sam!

It’s an honor to have been recognized as a Runner-Up for the ⁦@TheOfficialACM⁩ SIGSAC Doctoral Dissertation Award at CCS 2024 (⁦@acm_ccs⁩)! I’d like to thank my nominator and letter writers for this award, and my advisors, mentors, and collaborators during my PhD.

@luo_michael1234 @justinywong_ @brandontrabucco @bignamehyp @profjoeyg @rsalakhu Congrats Michael and Justin! What a big week for y'all

Excited to announce today that Stylus has been accepted at NeurIPS 2024 with Oral! Link: stylus-diffusion.github.io Special thanks to my collaborators and advisors: @justinywong_ , @brandontrabucco, @bignamehyp , Ion Stoica, @profjoeyg, @rsalakhu #AI #NeurIPS #neurips2024

Great work by @justinywong_ @luo_michael1234! I've been having a lot of fun voting on the video models.

Proud of what we came up with. The new Berkeley OS prelim balances breadth with depth by connecting key modern advances to ideas introduced in older papers. Syllabus: ucbosprelim.samkumar.org

@samkumar_cs @CS_UCLA @UCLA Congrats Sam! Wishing you all the best and much success at UCLA 🥳📚

I've wrapped up my postdoc and just moved down to Los Angeles. I'm super excited to start at UCLA CS (@CS_UCLA)! Today is my first day working in person at the @UCLA campus.

@profjoeyg Usage-based pricing makes sense AI infra (fine-tuning, generating tokens). AI Infra is like AWS, and is used to build applications (ChatGPT, Copilot). Fundamentally, infra and apps target different use-cases and could benefit from different price models.

@profjoeyg Interesting post. I think that monthly fees for AI products make a lot of sense because, as you mentioned, prices are predictable. Usage-based pricing increases complexity, and could harm adoption despite being more cost-effective.

What is the right pricing model for AI? Should it be a monthly fee or a flat rate per token? Do you pay extra for more knowledge? Three years ago, I was focused on server-less computing for AI and how to allocate inference engines. At the time, consumption based pricing was the future. Maybe it still is? @vsreekanti and I have been thinking about pricing models and we just published our latest thoughts: frontierai.substack.com/p/the-future-o…

Very cool to see another high-performance simulator for self-driving! GPUDrive addresses a key limitation of CARLA (simulation speed) and is even capable of simulating sensors like LiDARs.

@tianjun_zhang @AIatMeta Congrats Tianjun!!!

It has been really a rewarding journey since I joined the #LLaMA3 team @AIatMeta a little more than 2 months ago, and yet today we are releasing one of the world's best models! 🔥With the new license, we allow synthetic data generation from Llama to enhance your own model! Checkout our research paper on how we build this: ai.meta.com/research/publi…. Excited to see what we can build on top! 🫡

@kanaad Congrats 🚀🚀🚀

Interesting approach from Waymo to use Q/A for remote interventions. Perhaps this is used to improve driving models via RLHF?

@robertnishihara @mrry Hi Robert, considering all the lessons learned building Ray, are there any changes you would have made back when you first started the project? Personally, I always wondered whether the flexibility of dynamic task graphs would eventually lead to performance bottlenecks.

Ray originally started with just the "task" API for executing Python functions asynchronously (with some resemblance to systems like Dask, Celery, PySpark, etc). Actually, the system most closely resembling Ray's task API is CIEL (built by @mrry). usenix.org/legacy/events/… That said, a lot of AI is stateful, and the task API was just too limited, so pretty early on we ended up needing to add the actor API (essentially the ability to spin up a Python class as a little actor / microservice). The actor API was what really opened the floodgates and enabled Ray to support training workloads, online serving workloads, reinforcement learning workloads, and so on. Even Ray Data is built on actors despite data processing workloads being traditionally stateless.

@Michaelvll1 @skypilot_org Congratulations! 🥳

I am honored to share that our recent paper won the Outstanding Paper Award in NSDI’24! The paper explores the policy design of our SkyPilot managed spot for @skypilot_org: Can’t Be Late: Optimizing Spot Instance Savings under Deadlines It would not be possible, if it were not with the fantastic folks and advisors: @infwinston @ziming_mao @zongheng_yang Eric Friedman, Scott Shenker and Ion Stoica; and the whole SkyPilot team @skypilot_org

Spent the last few weeks working on this blog! When I first read the Ring Attention paper, I kind of get the concept yet not really. Diving into the details from the math to compute was incredibly rewarding for our understanding, and hope it be a fun read for you too!