enbao
60 posts
enbao retweetledi
Excited to share new ThunderKittens attention kernels that match or outperform Flash Attention 4 on Blackwell GPUs! Currently only supports QK192/V128 shapes, but more coming soon.
Check out the code here: github.com/HazyResearch/T…
Shoutout to the FA4 team for the algorithmic innovations and to @stuart_sul for the helpful discussions.
English
enbao retweetledi
(1/7) We're releasing ThunderKittens 2.0! Faster kernels, cleaner code, industry contributions, and new state-of-the-art BF16 / MXFP8 / NVFP4 GEMMs that match or surpass cuBLAS!
Alongside this release, we’re equally excited to share some insights we learned while squeezing every last TFLOP out of Blackwell:
(with @hazyresearch & generously supported by @cursor_ai)
English
wrote a personal piece on the importance of side quests and the point of it all -
enbao.me/posts/ocean
English
my first blogpost related to GPUs! this one looks at pyutils, a small but important part of the ThunderKittens library that allows kernels to be launched with PyTorch. enbao.me/posts/tk
English
@enbaoc Hey awesome blogpost! I def recommend you take cs149 on parallel computing gfxcourses.stanford.edu/cs149/fall25
we talk about many DSLs
English
The blogs are great. So refreshing seeing someone consume knowledge like this.
enbao@enbaoc
my first blogpost related to GPUs! this one looks at pyutils, a small but important part of the ThunderKittens library that allows kernels to be launched with PyTorch. enbao.me/posts/tk
English
day 1: arizmendi bakery → botanical garden → japanese tea garden → ushi taro ramen → conservatory of flowers → waymo → baker beach → batteries to bluffs trail → golden gate bridge at sunset → waymo → dalida → waymo → movie at the alamo drafthouse cinema → waymo
“paula”@paularambles
mom's visiting sf for a week. what are some good mom in sf activities (that won't result in her joining a seed-stage startup)
English
“we are squarely an ‘age of research’ company” hell yeah
Dwarkesh Patel@dwarkesh_sp
The @ilyasut episode 0:00:00 – Explaining model jaggedness 0:09:39 - Emotions and value functions 0:18:49 – What are we scaling? 0:25:13 – Why humans generalize better than models 0:35:45 – Straight-shotting superintelligence 0:46:47 – SSI’s model will learn from deployment 0:55:07 – Alignment 1:18:13 – “We are squarely an age of research company” 1:29:23 – Self-play and multi-agent 1:32:42 – Research taste Look up Dwarkesh Podcast on YouTube, Apple Podcasts, or Spotify. Enjoy!
English