light

This is the roadmap I've put together for the next few months. While I already have a background in ML, I've never really explored the performance engineering or CUDA side of things. It should be an interesting few months ahead.

anthropic no longer holds the heavenly mandate

Funnily enough, GPU-accelerated models already existed before AlexNet. Dan Cireșan and Jürgen Schmidhuber's group were already winning several vision benchmarks. Still have no idea why they didn't enter THAT ImageNet competition.

visualization of output-stationary CUDA GEMM with micro tiles (plus a little bit of pink floyd)

@srai009 Inkling also means a slight idea or an indistinct understanding, maybe the current stage of LLMs

@reprompting I just saw this post from the Thinking Machines team about the origins of the term "Inkling", a really apt name for the model as well :) x.com/luke_drago_/st…

More than anything, I like the name of this company, Thinking Machines. It's just so apt.

re-reading the original roofline paper today dl.acm.org/doi/pdf/10.114…

@goyalayus It's in article section, I think it's alright

@reprompting you probably want to add this to your highlights

Today's reading is Agner Fog's manual on optimizing software in C++ agner.org/optimize/optim…

@pc_desgn yeah got to learn the hard way

@reprompting GIFs on Twitter are encoded as H264 videos anyway, so no need to upload as GIF, the worst video compression format to ever exist.

I didn't realize you can't zoom in on a GIF, so I turned it into a video instead, with DOOM music

visualization of output-stationary CUDA GEMM

Today's reading is Anatomy of High-Performance Many-Threaded Matrix Multiplication. It extends the GotoBLAS approach to efficient multi-threaded GEMM on multicore processors. cs.utexas.edu/~flame/pubs/bl…

@junupark_ I feel like everyone should experience cycling along the Han River on a summer evening at least once.

touch water

This led me to find out just how "hidden" Vasily Volkov seems to be. There isn't much information about him on the web, especially considering how influential many of his papers and talks have been. 1. Better Performance at Lower Occupancy nvidia.com/content/gtc-20… 2. Benchmarking GPUs to Tune Dense Linear Algebra mc.stanford.edu/cgi-bin/images… 3. Use registers and multiple outputs per thread on GPU laurel.datsi.etsiinf.upm.es/_media/proyect… 4. Unrolling parallel loops nvidia.fr/docs/IO/116711… 5. LU, QR and Cholesky Factorizations using Vector Capabilities of GPUs netlib.org/lapack/lawnspd… 6. Understanding Latency Hiding on GPUs www2.eecs.berkeley.edu/Pubs/TechRpts/… 7. A microbenchmark to study GPU performance models dl.acm.org/doi/pdf/10.114… 8. Parallel computing experiences with CUDA cs.virginia.edu/~skadron/cuda_… 9. Fitting FFT onto the G80 Architecture people.eecs.berkeley.edu/~kubitron/cour… 10. Stencil Computation Optimization and Auto-tuning on State-of-the-Art Multicore Architectures csd.uwo.ca/~mmorenom/CS43… 11. Using GPUs to accelerate the bisection algorithm for finding eigenvalues of symmetric tridiagonal matrices eecs.berkeley.edu/Pubs/TechRpts/… 12. Building an Efficient Hash Table on the GPU sciencedirect.com/science/chapte… 13. Programming inverse memory hierarchy: case of stencils on GPUs laurel.datsi.etsiinf.upm.es/_media/proyect… And these don't even cover his contributions to computer graphics and computational neuroimaging. For someone whose work has had such a lasting impact across multiple fields, he seems remarkably low-profile.

@graffioh I found the early chapters to be exceptional. The later chapters focus more on applying the underlying principles in practical GPU programming. So for someone like me who's just getting started with this, it was a nice introduction.

@reprompting how it was? worth it?

Summarizing chapter 22-23 of PMPP (final chapters) These final chapters wrap up the book by looking at how CUDA and GPU hardware have evolved over the years, from seperate host/device memory models to unified memory, dynamic parallelism, cooperative kernels, better atomics,

@kernel_trick Just briefly going through things before fully committing, tbh

@reprompting (how do you find good theses worth reading)

todays read is vasily volkov's phd thesis on gpu latency hiding. it's quite long, so i think it'll keep me occupied for the next few days. www2.eecs.berkeley.edu/Pubs/TechRpts/…

@kernel_trick this was referenced here modal.com/gpu-glossary/p…

@reprompting how did you find this

the idea seems to be decomposing GEMM into smaller kernels and aggressively optimizing the lowest-level ones. if the microkernels are near peak performance, the rest is mainly organizing data movement, which carries over to overall GEMM performance.

todays reading is anatomy of high performance matrix multiplication by kazushige goto. cs.utexas.edu/~flame/pubs/Go…