LeetGPU

spoke to one of the builders at leetgpu.com, @therealkmans, about having an FP16 batched mat mul so i can try programming with tensor cores and he got it up in no time. i'll say it again, this is the best place to learn and build muscle memory on writing cuda

I locked in for 100 days to learn CUDA the HARD way and reached #1 on @LeetGPU. Detailed post explaining my learning journey coming soon...

Blog post for day 2 of doing one @LeetGPU challenge a day is live!! We go over writing a matrix multiplication kernel 🤯 We start with a naive solution and keep adding tons optimizations on top of it that leave us with the most efficient solution 🚀 arayz.dev/blog/leetgpu-m…

day 100/100 of GPU Programming Didn't write a kernel today. I spent the day reflecting. 100 days writing kernels and I didn't miss a single day, not one. On some days, I learnt to write new ones, some days I practiced kernels I've written before. I took on something my younger self would never have imagined taking on. In learning how to write kernels, I learnt how to learn. When I first heard about this challenge from @hkproj, I had zero knowledge about GPU Programming, I barely knew how a GPU looked or worked. I had no experience whatsoever with C/C++. GPU resources were scattered and scarce. And aside from that, my biggest challenge was being GPU poor. But that didn't stop me, I still decided to take on the challenge. I had no definitive path. If there's something I've witnessed is that If you solely decide to take something head on, resources will come your way. You don't have to have a very clear path, just get started. You'll get to see the road ahead as you go. I went from writing and profiling CUDA kernels on Google Colab to finding out about @LeetGPU which accelerated my progress exponentially because I now had some kind of definitive path writing and learning specific types of kernels. These past 100 days have been my best in terms of Programming. Through this journey, I made friends, got haters, got job offers and learnt so much. X is the best place to be, if you consistently put yourself out there, alike people will show up. 100 days ago I couldn't even write a vector add, fast forward 99 days later, I'm writing FP16 Matrix Multiplication Kernels via WMMA APIs. There are those who felt going on an 100 day challenge was obnoxious and performative. I didn't find that so, I pushed myself to the limit. I wrote kernels on days I felt burnt-out and depressed. I wrote kernels when I was busy with school and life, some days I didn't play football just so I can get a kernel done. In the process, I learnt to be consistent. I didn't do this for anybody by myself. Before @ludwigABAP and @0xmer_ noticed me and put me out there, I was basically showing my progress to myself(I had about 200 followers). Having more people inspired by my work made it even more fulfilling for me because I had people looking up to me. So for anyone who want to take on the challenge on study something for 100 days straight, I encourage you and for the people already in the process, keep going. Put your head down, ignore the voices and accelerate. You really can just do things!