Shekhar
905 posts
@indianspeedster
Distributed Training & Inference @amd | @nyuniversity ‘24 | Views are my own
This year, #Tandon students pulled out every trick in the #book at the annual #HackDibner Competition. Congrats to first place winners Trick Coders, computer engineering grad students Priyanka Bose & Chandra Shekhar Pandey. #NYUTandonMade bit.ly/3HufVTH
I’ve published a follow-up to my earlier post on speculative decoding. This one traces how the drafter has evolved, from Medusa’s parallel heads and EAGLE’s feature-level autoregression to Gemma 4 MTP, DFlash, and the recently released DSpark. What changed is not just improved draft accuracy. Newer designs also try to remove sequential draft passes, reuse target state, and avoid verifying suffix tokens that are unlikely to survive. ayushmansingh.com/posts/a-field-…
David Patterson is a Turing Award winner famous for his contributions to computer architecture. I interviewed him about his past work, thoughts on GPU/TPUs and career advice from half a decade of experience. In this episode: • RISC vs CISC historical debate • Comparing GPUs, TPUs, and CPUs • Is Moore’s law dead? • How to have a bad career • Advice to his younger self Where to watch: • YouTube - youtu.be/Pn4ZwlEh5nw • Spotify - open.spotify.com/episode/6mlRwX… • Apple Podcasts - podcasts.apple.com/us/podcast/the… • Transcript - developing.dev/p/turing-award… Chapters: 00:00 - Intro 00:42 - RISC vs CISC 12:51 - Compilers 17:38 - GPUs 23:07 - GPU vs TPU vs CPU 32:12 - Is Moores law dead? 38:04 - GPU benchmarks 41:40 - How to have a bad career 49:59 - Courage and optimism 55:56 - Advice for his younger self 58:15 - Outro
🎉 Great to see the @AMD team for bringing ROCm support to vime, the vLLM ecosystem's RL post-training framework. End-to-end RL post-training now runs natively on AMD Instinct MI355X GPUs. vime uses vLLM as its rollout backend, so on ROCm it inherits the full vLLM rollout stack with no separate code path. The @AIatAMD team validated the pipeline end-to-end, upstreamed the ROCm-specific fixes, and shipped a prebuilt container so you can skip building from source. What works today: - GRPO training - Colocated and async (non-colocated) train/rollout - Megatron-LM training + vLLM rollout backends - Qwen3 dense and MoE models On MI355X, Qwen3-8B sustains ~4,100 tokens/gpu/s, and the train-rollout logprob diff holds low and stable. 🔗 vllm.ai/blog/2026-07-1…
todays read is vasily volkov's phd thesis on gpu latency hiding. it's quite long, so i think it'll keep me occupied for the next few days. www2.eecs.berkeley.edu/Pubs/TechRpts/…
I published this a while ago, but I’m planning to write more frequently going forward, so I thought I’d reshare it. It’s my take on speculative decoding and an attempt to explain it from a different perspective than most existing writeups. If you haven’t seen it before, I’d love to hear what you think. ayushmansingh.com/posts/how-spec…
“Higher occupancy does not necessarily mean higher performance.” — Vasily Volkov Occupancy is one of the most misunderstood GPU metrics. I put together a practical guide on the occupancy math behind AMD MI355X: • Waves/CU • VGPR limits • SGPR limits • LDS limits • Real occupancy calculations Understanding these limits is the first step toward understanding when occupancy matters—and when it doesn’t. Blog 🔗 in comments.
