Amanzhol Salykov
131 posts
@salykova_
At the intersection of AI/ML systems, low-level GPU kernel optimizations and algebra @AMD
github.com/ROCm/TheRock/r… AMD has successfully transitioned/merged ROCm releases to the TheRock! congrats @AnushElangovan and the rest of the team, this is ROCm’s biggest win since 7.0 launched last year.
Better HIP kernels through synthetic data, multi-agent search, and reinforcement learning. See how researchers at @Stanford's Scaling Intelligence Lab are advancing code generation for AMD GPUs. scalingintelligence.stanford.edu/blogs/hipkerne…
LLMs are good at CUDA because the internet is full of it. But a model that gives you highly optimized CUDA may still struggle to write compilable HIP. We built a synthetic data pipeline with multi-agent search and post-trained a 14B open-source model with SFT + GRPO RL, leading to substantially better HIP compilation + correctness rates on AMD MI350X GPUs.
New mega work 📢 A fast BF16 forward attention kernel for AMD MI300X, written in HIP, not full hand-written assembly. Up to 1.26× vs AITER, and 1.37×–1.59× vs Modular MAX. But the interesting part is not just the benchmark. It is the engineering path. AITER v3 is written in hand-tuned GCN assembly. We wanted to see how far we could go while staying much closer to ordinary HIP. Technical details: The core technique is a middle path: one-instruction asm wrappers. We still choose the exact opcodes that matter, but leave register allocation and data-flow tracking to the compiler. This gives us instruction-level control without turning the entire kernel into a full assembly codebase. From there, the kernel is mostly a memory-placement and scheduling problem: • K is streamed into LDS and double-buffered • V is kept hot in L1 instead of staged through LDS • Q and accumulators stay in registers where possible • the CDNA3 pipeline is planned around 8 waves, 2 groups, and 2 carefully placed barriers • 3Q tiling increases data reuse • tail KV splitting keeps idle CUs from dominating long-context shapes One takeaway: high-performance AMD kernels do not necessarily require choosing between “let the compiler do everything” and “write the whole thing in assembly.” There is a useful middle layer where you give the compiler a structured framework and take control only where the hardware demands it. Code is open source under MIT. PR to SGLang is in review. moonmath.ai/cdna3attention/
