Core Automation

@_arohan_ @7Vivekrao God celebrates by training worlds deepest network

@7Vivekrao @CoreAutoAI wdyt

God is playing football against spain today. Good luck to spain! 🇪🇸

@DougDoesOps Who let the Doug out?

I have risen

Great results thus far from the 9 participants in the @GPU_MODE competition. 11B tokens and $500 in @modal credits thus far. Thanks to @damngamerz and everyone else for participating!

😀

Yes

Can a model run deeper at test time than it was ever trained to? And if depth becomes a loop instead of a stack, do we need better optimizers to keep training stable? @CoreAutoAI is collaborating with @tilderesearch on an optimizer x architecture competition: "One Layer Deeper" We picked a cursed problem y = x^(2^T) mod N where each squaring depends on the last. A vanilla transformer has a fixed depth so it falls flat on its face the moment T exceeds the serial compute it can do in one forward pass. A fun reminder that architectures have inductive biases. This isn't quite a nano-gpt speedrun experience and it's not a typical @GPU_MODE kernel competition either, it's closer in spirit to the LLM efficiency competition I worked on many years ago with @weiwei_msr and @Jisaacso in that it's open ended in a specific way. The UX is kernelbot like, your submission is a single python file that defines your model, optimizer and loss. You don't need a GPU to participate at all since we're reusing a queue based job system powered by @modal and @northflank There's tons of unexplored ideas and I'm really not sure what will win out but I'd be particularly excited to see people playing with new optimizers, adaptive depth and depth extrapolation. Since this is a new format, we'll work with the community to refine the rules for another week before we start for real. Submissions are open now!!

For people who care about linear algebra and kernels, we have last competition in the series, Choleski decomposition gpumode.com/leaderboard/776

onelayerdeeper.ai There were (and still are) kernel competitions. Now we want to present something new: optimizer competition. Show your skills, and learn along the way optimizing through hard problems.

Finally puzzle got solved

@DynamicWebPaige @marksaroufim @MillionInt @anmol01gulati @GPU_MODE Hi Paige!

🤩 Packed house for the @CoreAutoAI housewarming party! @marksaroufim and @MillionInt and @anmol01gulati and more. "Agents will be the biggest market in the world, with AI doing most of the work. And research is the engine of value creation." @GPU_MODE and kernels imminent!

Automation is leverage

LLMs are beautiful creatures

@SwishMoe Like Car research?

@CoreAutoAI Autoresearch

Agents + Research = ?

@DishwasherTag @_arohan_ That loser attitude, that loser premise makes no sense to me

@_arohan_ one thing is clear: @coreautoai most definitely did not wake up a loser

Just to be clear: we are not car auto. Not a car company.

@prateek_0041 Don't give up my friend. You can do it. CUDA is your friend.

i am so deep in the mud right now, i feel stuck honestly. CUDA or Inference Engines, Inference Engines or By hand implementation of a dummy engine with attention, KV cache. MLX or CUDA. I have touched all these topics but depth is lacking. I need to put in more time.

@willccbb such a missed opportunity...

robotics neolab called Chore Automation who’s building this

@_arohan_ 100%. I’m totally not Rohan, just an AI twin Nahor

I am definitely, most definitely not the person behind this twitter account. You must believe me.

So, Brain, what do you want to do tonight? The same thing we do every night, Pinky. Try to build AGI.

Hmmm