Matthew Nicely

Perplexity runs on NVIDIA. Nice breakdown from the team on how they’re using the CUTLASS Python stack to optimize their models for inference 👇

@SemiAnalysis_ Thanks for the shout out :) Folks can also checkout the cuDNN GPU MODE presentation on FP8 & MXFP8 attention optimizations - youtube.com/watch?v=HcnybH…

For the past 12 years, cuDNN has been completely closed sourced (besides the .h files), until this week! OVER 20 MoE kernels & NSA sparse attention kernels from cuDNN has been open sourced! Great work to @manicely6005 & the rest of the team on seeing that parts of NVIDIA are moving towards open kernels! open source kernels drive innovation! (1/3) 🧵

NVIDIA cuDNN engineers are diving deep into high-performance computing — sharing insights on peak performance with FP8 & MXFP8 attention. 📅 May 5th, 2026 | 🕛 12pm PST 🎥 Watch live on YouTube: lnkd.in/ey6YCZV3 #GPUMODE #cuDNN #AI #FP8 @cudagdb @marksaroufim @blelbach

@maharshii Does it need to output C++ or would Python work as well?

@manicely6005 it would be nice to have a python api for defining a gemm op with custom EVTs and generating C++ / directly compiling from that (for quick prototyping, instead of writing everything by hand in cutedsl)

does nvidia cutlass have a python API for generating GEMM kernels (with collective epilogue) like we do in C++? they do have something called PyCutlass but that seems to be deprecated. note: i'm not talking about CuTeDSL where the user needs to write the whole kernel.

@GPU_MODE More kernels will be rolling out soon.

Incredible release

@dylan522p This has already been approved and several are in the process of being rolled out 🙂. We’ll do a better job of communicating soon.

NVIDIA has now open sourced their trtllmgen MoE kernels! Great to see that parts of NVIDIA move towards open kernels! open source kernels drive innovation It is now time for SemiAnalysis to nag NVIDIA to open source their trtllmgen attention kernels too!

This Saturday 10:00 AM PST, the last talk of the year before we resume again on Jan 3. NVIDIA has made a profound change to its programming model with cuTile and TileIR. They've given some shorter talks online but this will be the first deep dive youtube.com/watch?v=sjkEUh…

btw today at 3pm PST (in ~4 hours) we're having Vicki Wang from NVIDIA giving a @GPU_MODE talk on CuTe DSL, its features, and how to use the most of it if you're currently competing in the NVFP4 Blackwell competition this will be very helpful, but it's open to anyone!

CuteDSL 4.3.1 is here 🚀 Major host overhead optimization (10-40µs down to a 2µs in hot loops_, streamlined PyTorch interop (pass torch.Tensors directly, no more conversions needed) and export and use in more languages and envs. All powered by apache tvm-ffi ABI

Today we are releasing our first public beta of Nsight Python! The goal is to simplify the life of a Python developer by proving a pythonic way to analyze your kernel code! Check it out, provide feedback! Nsight Python — nsight-python docs.nvidia.com/nsight-python/

Ready, Set, Go! 🏎️ Create something amazing at our Blackwell NVFP4 Kernel Hackathon with @GPU_MODE. 🎊 🏆 Compete in a 4-part performance challenge to optimize low-level kernels on NVIDIA Blackwell hardware. 🥇 3 winners per challenge will receive top-tier NVIDIA hardware. The fastest of all earns the grand prize - a @Dell Pro Max with GB300. And some will earn a pass to #NVIDIAGTC in San Jose in 2026. Register now 👉 luma.com/9n27uem4

1,000 registrations so far!

@elliotarledge Just curious, what tool you used to write the nvfp4 gemm?

timelapse #89 (12.5 hrs): - got single gpu nvfp4 gemm @ 5.2 PFLOPS working reliably (sm100) - solved ampere/hopper gemm kernel from scratch issues - split kernel optimization chapter into: - gemv, softmax, layernorm, topK, gemm (fp32 only cuda cores) - gemm (tf32, fp16, bf16, fp8, fp4) - cutting sugar made me feel great in the morning but killed me later in the day so went to bed super early - more hyperengineering tomorrow (ordered 96 diet cokes)

@anneouyang I wonder how it would do on the cutedsl??

i asked gpt to rewrite a matmul kernel in cute

Shoutout to @colfaxintl for formalizing the math behind CuTe layouts. Their research connects CUTLASS layout operations to solid category theory foundations. Valuable work for anyone building custom kernels. Blog: research.colfax-intl.com/categorical-fo… #CUTLASS #GPU #CUDA

CUTLASS 4.1 is now available, which adds support for ARM systems (GB200) and block scaled MMAs

Speed-of-light kernels for Blackwell decode now in flashinfer. Give em a spin.

CUTLASS: Principled Abstractions for Handling Multidimensional Data Through Tensors and Spatial Microkernels | NVIDIA Technical Blog developer.nvidia.com/blog/cutlass-p…

CUTLASS 3.x: Orthogonal, Reusable, and Composable Abstractions for GEMM Kernel Design | NVIDIA Technical Blog developer.nvidia.com/blog/cutlass-3…