Sharon Zhou

It's here: We just hit superhuman performance on AI kernel optimization! Real customer models & production settings. Not toy problems (what I typically see). This is the year that Claude writes its own kernels, Codex its own kernels, for every new GPU that it wants to run on -- something that takes months to port between GPU generations today. This has a massive impact to scaling intelligence. More compute means getting the next frontier model sooner.

New course: Transformers in Practice. You'll get a practical view of how transformer-based LLMs work, so you can reason about their behavior, diagnose problems like slow inference, and make smarter decisions about deployment. This course is built in partnership with @AMD and taught by @realSharonZhou. You'll see how transformers generate text one token at a time, how the model decides which earlier words matter most when predicting the next one, and how techniques like quantization speed up inference on GPUs. This is not a video-only course; interactive visualizations throughout let you play with these concepts and build intuition that sticks. Skills you'll gain: - Understand why LLMs hallucinate, and RAG and chain-of-thought shape what they generate - Look inside the model to see how attention and layers combine to predict the next token - Diagnose inference bottlenecks and learn the techniques that speed up transformers on GPUs Join and understand what's really happening inside your LLMs: deeplearning.ai/courses/transf…

Excited to share a new, practical course on Transformers! In partnership with AMD and DeeplearningAI Enroll free :) deeplearning.ai/courses/transf…

Slow inference. Hallucinations. Costs that don't scale. The parts of LLMs you can't see are the parts that bite you. Build the intuition to debug them, in our new course with @RealSharonZhou and @AMD: Transformers in Practice. Enroll here: hubs.la/Q04g8KSv0

@venkat_systems @LisaSu @SemiAnalysis_ @AnushElangovan Thankfully for myself, only roc-stars like Anush get that type of glory

@realSharonZhou @LisaSu congrats ! 🚀 will you also appear now in @SemiAnalysis_ memes along with @AnushElangovan ?

Happy to share I’m expanding my role to report directly to @LisaSu!

@GrahamsonT @LisaSu @AnushElangovan Nah, no one's got anything on Anush. He's a roc-star.

@realSharonZhou @LisaSu Congratulations. @AnushElangovan you got competition!

@corentinanjuna Yes end to end- it’s already operating at the model/framework level

@realSharonZhou The real deal would be to let the agend improve the whole stack from driver to libs to kernel.

codegen is cheap now. performance isn’t: most generated kernels are kinda mid. iteration and feedback are missing for both the agent and RL env layers. so we're open-sourcing Apex: an end-to-end agent using Claude Code + Codex to effectively optimize AMD kernels, instead of one-shotting them github.com/AMD-AGI/Apex

@Tahseen_Rahman yeah agreed on the 100% autonomy case - tho I guess for Codex, if it's an issue that a knowledgeable engineer can fix and still get gains, it'd be faster to solution as an agentic partner

@realSharonZhou Speed vs. quality is the wrong framing. Codex shipped faster but unusable. Claude shipped slower but production-ready. The real metric: time to working solution. Not time to first attempt.

Mood: agents optimizing kernels Claude won on kernel optimization: gemm_bf16 at 1.19x vs Codex's 0.94x. Codex was faster (~1.3h vs ~3.4h) but produced no reinjectable optimizations. Claude used torch.mm (hipBLASLt) as a drop-in replacement for the custom Triton kernel. For Codex, shape mismatch caused slight regression. Still improving, open sourcing soon --- AMD-AGI team (Sina Rafati, Emad Barsoum, and many more)

@Haniel_Ulises Heard of triton and gluon?

@realSharonZhou He does it in p*thon, so it's not impressive

It's here: We just hit superhuman performance on AI kernel optimization! Real customer models & production settings. Not toy problems (what I typically see). This is the year that Claude writes its own kernels, Codex its own kernels, for every new GPU that it wants to run on -- something that takes months to port between GPU generations today. This has a massive impact to scaling intelligence. More compute means getting the next frontier model sooner.

@gonzaleshvili also memories I teach the post-training course on DeeplearningAI, and writing a book on post-training/RL - so Claude and I chat a lot about it… Also this Xmas I got obsessed with loss landscapes and vibe codes some loss landscape video games - and that shows up here…

@gonzaleshvili Yes

Wait this is so good Asked Claude to make a video on RL post-training

“Thought pruned by optimizer” got me

@ysu_ChatData I’m worried about correctness checks being the bottleneck due to reward hacking and the fact that many existing tests are written for humans; less worried about benchmarking speed as we’ve been optimizing pieces there

This is a big deal. If models can write kernels, the hard part becomes correctness checks plus fast benchmarking and regression tracking across drivers and GPU generations. Curious what your verification loop looks like, do you use reference implementations, property tests, or differential testing against vendor libs?

@macro_guru rocm.blogs.amd.com/software-tools…

@realSharonZhou Focus on composability of inference optimizations for ROCm stack in order to defeat the CUDA moat. (wide EP + disagg + FP4 + KV offloading + more) for maximum efficiency/scaling on huge models. AMD/ROCm lags in making those combinations smooth

@Strakyo I agree that's ideal and we do see generated kernels that generalize, but also the "real only if" part is not necessarily true in the frontier world

@realSharonZhou Kernel wins are real only if they hold across model families. What are the p95 speedup and numerical-drift deltas on held-out production workloads?

@sebastianboehle here's the tool/mcp that helped (3000x more token efficient at regular profiling/benchmarking) github.com/AMD-AGI/Magpie working on open-sourcing the end to end pipeline you see in the gif

@realSharonZhou Can you share a preprint or GitHub repo? Am also working on a fork of cuda agent from bytedance on an optimized kernel writing agent. Would love to benchmark