Zeke

@clattner_llvm Does the solver recover the exact same schedule as flash attention? It would be cool if there were multiple optimal solutions or if it found something faster.

Pipelining AI kernels is required to get full perf/utilization out of modern chips. However, no one has been able to crack "full control over the hardware" without "having to micromanage it". Let's crack this open: kernel authors deserve a powerful scheduler they can control. 💪

@_xjdr What do you like about them?

gpt-oss and gemma-3n are such good architectures . i didnt really appreciate them until i really put them to the test through rigorous ablations

@elliotarledge @AlpinDale @HazyResearch Did you write this with a llm?

After diving head first into the deep end of squeezing every last drop out of inference megakernels, I decided to write a book about my journey, as well as how others like @AlpinDale and @HazyResearch architect their megakernels on hopper and blackwell. This assumes comfort with CUDA and LLM inference. elliotarledge.gumroad.com/l/grokking-meg…

@alz_zyd_ the normie's confusion with calculus is totally understandable. suppose i told you in 1650 that, in 1660, i could produce a closed form equation for the area under any continuous function between two points. you'd obviously think i'm a complete nutcase

The normies' confusion with AI is totally understandable. Suppose I told you in 2015 that, in 2025, gradient descent would produce black magic billionparameter bots which could think and talk and solve math questions. You'd obviously think I'm a complete nutcase

@GregKamradt @arcprize

Open questions with Gemini Deep Think and @arcprize: - Why doesn't it get 100% on ARC-AGI? We're trying to understand failure modes, need to inspect tasks more - Why a slight jump in ARC-AGI-1, but a 2x SOTA in ARC-AGI-2? Here are tasks that stood out in our early testing: