llm_enjoyer
1.4K posts
Once you learn about this, you start seeing it everywhere.
you guys are being way too hard on the anthropic video i mean come on are you really freaked out by this?
Introducing Flash-MSA, the world’s first open source sparse attention training kernels optimized for extreme context lengths. 4 simple kernels, 400%+ speedup over dense flash attention at long context ⚡
Introducing Flash-MSA, the world’s first open source sparse attention training kernels optimized for extreme context lengths. 4 simple kernels, 400%+ speedup over dense flash attention at long context ⚡
try starting every day with a 30 minute walk, don't bring your phone, and aim to be like a 75 year old peaceful man who walks slowly, looks at the surroundings, and feels the wind like i'd guess 50% of people in their 20s never again put their full attention on a tree
1.58 bit model quantization tutorial
"yeah no our context compaction does not degrade the performance of or otherwise affect the model at all"
you’ll get mad at me for saying this…but cloud gaming is so obviously more economically efficient than physical hardware I think it’s going to be the default soon. your home console / pc is idle 90%+ of the day. meanwhile, data centers targets what, 5%, maybe at worst 10% idle. every second a cloud gamer isn’t gaming, that hardware is being used for someone else, training, etc. I think there should be a new measurement, something like cost-per effective FLOP hour that takes into account the TCO + effective utilization. If a gamer spends $500 on a GPU, uses it for 3 years, but it’s only fully active ~5% of that period…the cost-per relative FLOP hour is crazy high! Meanwhile, a $50,000 datacenter GPU might have a *LOWER* cost-per FLOP hour just because the effective utilization is 90+%.
