bikram_

This repository is a public log of my learning, experiments, and projects as I dive deep into: - GPU architecture - CUDA programming - Memory hierarchies - Parallelism - Acceleration for deep learning and scientific computing github.com/bikrammajhi/10…

Dissecting Nvidia Blackwell - Tensor Cores, PTX Instructions, SASS, Floorsweep, Yield Microbenchmarking, tcgen05, 2SM MMA, UMMA, TMA, LDGSTS, UBLKCP, Speed of Light, Distributed Shared Memory, GPC Floorsweeps, SM Yield newsletter.semianalysis.com/p/dissecting-n…

@jedmaczan Great stuff 👍🏾

I built a tiny-vllm in C++ and CUDA - paged attention - continuous batching - educational - 100% human-written™ And now I writing a course where you will build your own vLLM yourself. Still work in progress, I'll finish by the end of April. All for free ofc, just a GitHub repo

for my next adventure, @michael_trbo and I will be working together to build a tinyLPU! for our first checkpoint, we reinvented the MXM: the language processing unit's matrix multiplication engine. here's how we did it

New blog post: "MXFP8 GEMM: Up to 99% of cuBLAS performance using CUDA + PTX": danielvegamyhre.github.io/2026/03/29/mxf… As someone who works on MXFP8 training, I was interested in deeply understanding GEMM design for this numerical format. In this post, we write a MXFP8 GEMM with CUDA + PTX, and iteratively optimize to reach cuBLAS-like performance (for some shapes!). It includes technical deep-dives into all the weird constraints and design challenges introduced by MXFP8. My brain is absolutely fried on CUDA+PTX now, so time to move onto other things (CuTeDSL?) - but in the meantime, time for me to go touch some grass

Followed this for a fun Saturday afternoon and... 1.15x faster prefill 🔥 The speedup comes not only from the math trick but also the WY kernel fusion For training short sequences caching UW caused too much overhead, so only did that >2048 ctxlen. 1.05x training🚀 Link below ⤵️

“This Kernel Was Faster Yesterday” — In Pursuit of High-Fidelity GPU Kernel Benchmarking standardkernel.com/blog/in-pursui…

PyTorch → CUDA “bro wrote 500 lines just to rediscover matmul” CUDA → PyTorch “you don’t optimize you just pray torch.compile does something”

Finally, "A Visual Guide to Mixture of Experts" is the last in the reading list. It can be applied to both Transformers and Mamba architectures, so it's best to leave this to last. You can view this as an extension of Chapter 3 and the Mamba guide. newsletter.maartengrootendorst.com/p/a-visual-gui…

Happy to introduce my video on this alternative LLM architecture, Mamba and State Space Models! I wanted to do it for a while now and finally found the time to work on animating my visual guide. Expect many, many, many visuals! Link in comment 👇

Really exciting to see KV-cache compression getting attention. A similar bottleneck shows up beyond LLMs: for world models and autoregressive long-video generation, KV cache can quickly dominate memory and limit long-horizon consistency. Our recent work, Quant VideoGen, explores training-free 2-bit KV-cache quantization for video diffusion models, achieving up to 7.0× KV memory reduction with <4% latency overhead. Link: arxiv.org/abs/2602.02958

I/O-aware Linear Attention arxiv.org/pdf/2312.06635 arxiv.org/pdf/2405.17381 arxiv.org/pdf/2401.04658

@KrishRShah 🤯🤯

You can turn your laptop into a trumpet

Open-sourced IKP: Intra-Kernel Profiler for CUDA kernels. Most GPU profilers tell you what happened at the kernel level. IKP shows what happened inside the kernel, for developers, and for agents. Repo: github.com/yao-jz/intra-k… #GPU #Profiling #CUDA

Cool pure Python implementation of CuTe layout algebra: github.com/facebookresear… -- with it, it only took a few minutes for Claude to make all of the CuTe paper arxiv.org/abs/2603.02298 have executable Python code with it too github.com/ezyang/cute-in…

Groq Language Processing Unit (LPU) at home arxiv.org/abs/2408.07326

@nash_c_brown Awesome

Excited to share new ThunderKittens attention kernels that match or outperform Flash Attention 4 on Blackwell GPUs! Currently only supports QK192/V128 shapes, but more coming soon. Check out the code here: github.com/HazyResearch/T… Shoutout to the FA4 team for the algorithmic innovations and to @stuart_sul for the helpful discussions.

Roadmap: Understanding GPU Architecture from Cornell cvw.cac.cornell.edu/gpu-architectu…

Wrote a deep dive on implementing a language model from scratch in JAX and scaling it with distributed training! If you’re coming from PyTorch and want to see how the same ideas look in JAX, or just want a hands-on intro to distributed training, check out this blog post: chuyishang.com/blog/2026/jax-… Comes with code + an assignment and test cases so you can follow along!