Pete Cheslock

Hey Boston Friends. Join us next week during Boston Tech Week to learn more about llm-d, open source distributed inferencing on kubernetes. Special thanks to @RedHat and @Google for helping to plan and sponsoring this free event! luma.com/eqbc1gxq

Boston Friends! Come and join us for what's shaping up to be a great event! May 28th at 5pm - Google Cambridge office.

We’re introducing an agentic OS prototype! Check out the demo and start building.  sprou.tt/12sUilhIwa6

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗔𝗽𝗿𝗶𝗹 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁! Our goal with this newsletter is to give a clear, community-driven view of what’s happening across the model serving ecosystem, including updates from projects like @vllm_project, KServe, @_llm_d_, @kubernetesio, Llama Stack, and more. 👉 Check out the April newsletter here: inferenceops.substack.com/p/state-of-the… 👉 Subscribe to get future issues in your inbox: inferenceops.substack.com 🚀 Thanks to everyone who subscribed so far! Kudos to all contributors to this edition! Francisco Arceo, Pete Cheslock, Jooho Lee, Pierangelo Di Pilato, Nir Rozenbaum, Yuan Tang, Wentao Ye, Sasa Zelenovic

vLLM meetup is coming to Boston on March 31! Workshop + evening sessions covering: - @vllm_project update - Model compression and speculative decoding - Agentic AI with vLLM - Distributed inference at scale with @_llm_d_ and Kubernetes Pre-event workshop at 3:30 PM: Deploy

@MartinGree93211 @RedHat Exactly! llm-d revolutionizes apps by treating inference as a first-class cloud-native workload. Features like disaggregated serving and inference-aware routing mean you get much higher throughput for less GPU spend. It’s all about making performance portable and accessible.

@RedHat Great to see Red Hat's work on llm-d for optimized model serving! This could lead to faster results and reduced costs. Pete, any insights on how this could revolutionize our AI applications? 😀

Red Hat is working with industry leaders to develop llm-d, an open-source project that optimizes how models are served to your users. By routing requests to the most efficient GPU and separating prefill from decode, you get faster results for less spend. Check out Pete Cheslock's quick overview of how llm-d is changing the game for Kubernetes-based AI: red.ht/3PbTkkP #KubeCon + #CloudNativeCon

ICYMI: llm-d is officially a @CNCF Sandbox project! 🚀 We’re evolving #Kubernetes into SOTA AI infrastructure through a powerhouse coalition including @RedHat , @googlecloud , @IBMResearch, @NVIDIA, @MistralAI, @huggingface , and many more. cncf.io/blog/2026/03/2…

Wondering what llm-d is? It's the open source project simplifying LLM deployment! Run any model on any accelerator, on any cloud. #llm-d #OpenSource #AI #Kubernetes #KubeCon

It’s official: llm-d has joined the @CNCF! 🚀 Our mission to evolve Kubernetes into SOTA AI infrastructure just got a massive boost. This milestone belongs to our amazing community. Thank you for building this with us. 💜 We’re just getting started! 🔗 cncf.io/blog/2026/03/2…

.@Redhat is contributing llm-d (@_llm_d_) to @cloudnativefdn as a Sandbox project. This isn't just a hand-off of code. It’s a commitment to making high-performance #AI serving a core, portable capability of the cloud-native stack. #KubeCon #CloudNativeCon sprou.tt/1tpPWTpSa85

For all my local Boston friends. If you are interested in vLLM/llm-d and inference at scale you should join us!

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗠𝗮𝗿𝗰𝗵 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁! We launched our newsletter publicly last year to share our contributions to upstream communities from our @RedHat_AI teams. We’ve gained over 𝟭𝟯𝟬𝟬 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲𝗿𝘀! Our goal with this newsletter is to give a clear, community-driven view of what’s happening across the model serving ecosystem, including updates from @vllm_project, KServe, @_llm_d_, @kubernetesio, and Llama Stack. 👉 Check out the March newsletter here: inferenceops.substack.com/p/state-of-the… 👉 Subscribe to get future issues in your inbox: inferenceops.substack.com 🚀 Thanks to everyone who subscribed so far! Kudos to all contributors to this edition! @franciscojarceo, Pete Cheslock, Sean Condon, Jooho Lee, Pierangelo Di Pilato, Ran Pollak, Nir Rozenbaum, @TerryTangYuan, Wentao Ye

We’ll cover all of this and more during our distributed inference meetup in New York City on March 11, 2026: luma.com/0crwqwg4

LFG!!!!

DON'T TOY WITH MY EMOTIONS

If you are in NYC next Wednesday, come and join us to learn how to scale AI Inference on Kubernetes with the llm-d project.

What’s on the agenda for next Wednesday's NYC meetup? 🛠️ Intro to llm-d 0.5 ⚡️ Distributed LLM serving on AMD 🧠 Lessons scaling Wide-EP and MoE 💾 KV-cache offloading & prefix scheduling Join us building the future of open-source inference. Details: luma.com/0crwqwg4

Join us next week in NYC with the llm-d community for a deep dive into distributed inference. We’re talking llm-d 0.5, scaling MoE models, and KV-cache offloading. If you're building LLM infra, don't miss this. 📅 March 11th 📍1 Madison Ave Register: luma.com/0crwqwg4

RT @TerryTangYuan: We'd like to announce that @kubernetesio WG Serving has succeeded and will be disbanded! Thank you everyone who have pa…