Adarsh

Serving GLM5.2 NVFP4 Agentic Workload with SGLang: How We Reached 500 TPS on 8xB300 at bs=1 In this deep dive, we explain how SGLang reaches 500+ tok/s/user at bs=1 on 8xB300, with 18 to 34% higher single-user interactivity within two weeks since day-0, and 6 to 11% better peak throughput at high concurrency, benchmarked on a real multi-turn agentic coding workload. Our new TopK-V2 kernel is 2.33x faster at 80K ISL, scaling to 10.17x at 1M ISL, keeping interactivity essentially flat out to 1M tokens. Part of the story is the architecture itself. GLM-5.2 applies IndexShare to its DSA layers and ships a stronger MTP head reusing IndexShare and KVShare. The rest comes from our serving optimizations. Special thanks to @NVIDIAAI for the help in day-0 support of GLM-5.2 NVFP4, and to @Zai_org for IndexShare in SGLang!

[Hugging Face ML Club India] This is tomorrow (15th July 2026). People who have been accepted to the event, please be on time. We do not want you to miss the Google DeepMind talk that we have in store for you! You will spot @RisingSayak @adarshxs @adithya_s_k and me at the event. (We don't bite, so come and say hi)

Had a great time here, got to learn so much in such a concentrated time frame. The room was really well curated while talks were about actual real world problems in AI infra. Thanks @adarshxs, @VinayakGavariya. @huggingface X @sgl_project X @SarvamAI

We organized AI Infra Day with @sgl_project and @huggingface which was a hit. 1000 registrations in 48 hours. 250 people in the room. a strong technical crowd, mostly ml/ai engineers, researchers, applied scientists, software engineers, and founders. people joined from microsoft, adobe, qualcomm, juspay, red hat, zscaler, google, amd, amazon, and more. special thanks to @alxnails for coming to India and doing this. looking for more such events!

Remember, the coolest event of the week is happening on the 15th. Those who have got their registrations confirmed, we're looking forward to building stuff with you! See y'all on D-Day!

@kts_o7 @huggingface @SarvamAI @tokenbender Thank you for attending!

AI infra day - @adarshxs from @huggingface was a great host. @SarvamAI s engineers gave a wonderful insight on how to approach hosting and serving models at scale. @tokenbender had a really cool topic but unfortunately I couldn’t grasp it much. Overall a productive day !

SGLang’s first community meetup in India is happening now 🇮🇳 Huge thanks to our amazing co-hosts @SarvamForDevs and @huggingface 🙌

@thepushkarp Lfgggg

i’ll be here tomm, looking forward to learning from all the super interesting talks

would be there at this crazyy event (SGLang x Sarvam x HF) looking forward to meet some amazing people who are building in the space and learn :)) lmk if you’d wanna meet up

@utkarsh_k1c @akosmicfool Hey, we are carefully sending out invites if someone opts out last minute. Please send me their mail id so I can confirm an invite for them.

@adarshxs yo still giving invites ? my friend @akosmicfool seeking one...

12 noon. Tomorrow. Researchers, engineers, founders, and investors. All under one roof.

@cneuralnetwork @dejavucoder Mail id?

@dejavucoder still waiting :(

will be here tmrw in pink and black clothes (pfp style)

@dejavucoder We should make this the dress code

The transformers modeling backend in @vllm_project is now as fast as vLLM itself. What does this mean? Adding a model to transformers now gets you access to 100% of the perf you would have natively. Do it once, train it anywhere, infer it everywhere, deploy it on all hardware.

Big news from @hmellor_ + @huggingface team🙌! In v0.25.0 the Transformers modeling backend hits parity with hand-written vLLM models. Now 450+ transformers architectures run in vLLM at native speed with zero porting. Integrate once with transformers to get vLLM's fused kernels, torch.compile, and CUDA graphs for free. Read about the changes below 👇

@alokbishoyi97 @EVO__HQ Bring them to Infra Day!

@Teknium GOAT

Super proud to say that the team and I put almost all our effort into resolving every P0 and P1 issue and PR in the entire Hermes Agent repo over the last week and a half, and as of 5 minutes ago, after an all-nighter, we've resolved 100% of them all! Extremely special shoutout to @Kshitijjkapoor who's been burning them away with me day and night! We aim to keep all of them 0 forever from here 🫡🫡

Bangalore AI folks, this month is packed, ICYMI @huggingface is hosting 3 different events in bangalore this month > (Jul 11) AI Infra Day - SGLang × Sarvam x HF > (Jul 15) HF ML Club - DeepMind × HF > (Jul 25) HF ML Club- PyTorch × HF × Red Hat I'll be at all three events and will also be giving a talk on RL at the PyTorch × Hugging Face × Red Hat meetup. Looking forward to meeting you all!

I'll be in BLR for this event and sticking around for a few days afterward. Looking forward to catching up with everyone who's attending! I'm also hoping to meet technical folks, founders, and operators who have experience with either (or both) of these: 1 - Transitioning from a horizontal product to a vertical solution. 2 - Building and scaling enterprise software sales. If that's you , or someone in your network , I’d love to connect while I'm in town.