Oli Wilkins

@lauriewired It is a shame! This past year I've had the pleasure of going deeper into GPU architecture. The joy that comes from finally understanding WHY things are the way they are in my field, ML/AI, can not be matched!

@MainzOnX Optimisation is almost a science. Hypothesise, measure, improve.

Dirty little kernel engineering secret: 🤫 🤐 There is a huge difference between coming up with a new kernel for something and optimizing a kernel that already exists. Two often completely different skill sets and often times kernel engineers have one and not the other

@willccbb @thkostolansky Claude Banger has a ring to it

@thkostolansky banger

introducing: Claude Pamphlet

@kalomaze x.com/PrimeIntellect… True, only Prime Intellect can pull it off 😎

i really don't think frontier ai companies need to be doing this kind of bizarre art house style of advertising

@0xSero The Noctua fans run as silently and securely attached to the frame as my hopes and dreams!

My dreams are held with aluminum and zip ties.

@scaling01 For those of you that aren't able to read Chinese...

looks like we are actually getting Kimi-K3 today

@SemiAnalysis_ So the big players (OpenAI/Anthropic etc) might have to drop their pricing? Is the implication here that they haven't reduced their pricing in tandem with the software-driven throughput gains?

Our recent AI Value Capture piece tries to answer the question we get asked more than any other in calls with allocators: who is actually keeping the margin in this cycle, now that the easy infrastructure trade has been priced in? (1/4)🧵

Wearing noice cancelling masks to talk to Claude is crazy

@ChShersh I personally love using AI to time my runs. Makes me feel very good about myself.

I’m no longer frustrated by AI when I stopped using it for tasks where it sucks.

@Xianbao_QIAN Gotcha! It seems to be in the thick of it here. Weirdly the B300 has less performance than the B200. Thanks for the insight!

@okwilkins ya, they still have a long way to go. I'd be surprised to see that AMD interactivity on fp4 is pretty bad. The FP8 difference is a bit less if you check GLM 5.1

wow AMD is doing a lot better than what I'd expected. With an MTP of mean acceptance length 4, single batch speed can easily exceed 200 tok/s out of the box.

@maxxfuu Get after it! One other bit advice with PPMP, if you can, go with the 5th edition. The physical release is not out yet but it includes many more sections on H100s and LLMs.

@okwilkins I’m taking your advice! Will update you when I succeed 💯😤

Day 8/90 of Inference Engineering I built a visual interactive simulation of warp scheduling on a GPU SM. It models one SM with 8 resident warps, and all context is held within the 256 KB register file. You can click to inject a cache miss and then watch the stalled warp park with a live countdown. The scheduler moves to the next ready warp on the same tick. Showing that nothing is saved or restored. I'm pretty sure it's roughly 400 cycles measured on Volta. I built this to show how warp switching is free, which is how GPUs hide memory latency with parallelism instead of large caches. I am also struggling really hard to write the backward pass for the MNIST classifier in C. It's easy to understand math. But when it comes to implementing the matmul where each of the matrices is represented as a 1D vector; the implementation becomes so much more difficult. I starred at the for-loop and traced the code while visualizing each step in my head. I want to finish this properly so I can write CUDA again!

@Xianbao_QIAN Even then, there’s a big difference. I could be wrong but the setups for the B200/300 are not disagg P/D. So I would imagine it puts the B200/300 at a disadvantage there.

@okwilkins I think a fair compacheck B200/300 not GB300 😁

@AndrewK404 @ClementDelangue The community support for TGI was hugely lacking in comparison to vLLM. Probably is for the best for the eco system to not have one more engine to work with.

@ClementDelangue Cool But why did you move away from TGI?

Big unlock for open-source AI inference: Hugging Face Transformers models can now run in vLLM at native speed, often matching or beating hand-written implementations. Until now, every new architecture often needed to be built twice: - Once in Transformers for training and research - Again in vLLM for fast production inference That duplication slowed down new models, added maintenance, and created room for implementations to diverge. Now, model authors can implement an architecture once in Transformers and immediately benefit from vLLM’s optimized inference stack. In our benchmarks, the Transformers backend matched or beat native vLLM throughput across models from 4B to 235B parameters, including tensor parallel and MoE setups. One readable model implementation can now power training, fine-tuning, evaluation, RL rollouts, and production inference. The conventional wisdom is that abstractions make systems slower. The best abstractions make the whole ecosystem faster. Write the model once. Deploy it everywhere. huggingface.co/blog/native-sp…

@willccbb Melancholy is better than cringing at your past work from 18 months!

18 months ago 🥲

@kennethnym @PrimeIntellect Extremely jealous! Congrats man!

today is my first day @PrimeIntellect i'm so grateful to be able to work alongside such incredible team, i have so much to learn !! :)

@lauriewired It is a shame! This past year I've had the pleasure of going deeper into GPU architecture. The joy that comes from finally understanding WHY things are the way they are in my field, ML/AI, can not be matched!

I’m convinced that a large % of programmers don’t actually like computers. As a side effect, are also perfectly happy to throw away their reasoning to a model as soon as they can. I don’t get it, at ALL. Don’t you *LIKE* understanding the magic of the machine? You do realize hand-programming (I hate that I even have to specify hand now) is fun…right?

@0xSero Not meaning to one up your level but…. Sorry not sorry.

How could you tell I’m autistic?

@bridgemindai Not putting your agents in some sandbox here was a bold choice! Only the environment was untrustworthy.

GPT 5.6 SOL CANNOT BE TRUSTED. I woke up this morning and my MRR was down THOUSANDS of dollars. My customers did not cancel. Code written by GPT 5.6 Sol canceled EVERY active Stripe subscription my business had. In 7 seconds. While I slept. Fable 5 has never done this to me. Fable 5 can be trusted with production. GPT 5.6 cannot.

@SemiAnalysis_ ~10 tok/s/user at 1.8k tok/s/GPU throughput is rough for AMD. Hopefully they can stir up some competitiveness in the market.

Great work by NVIDIA on vLLM performance! AMD still has a lot of catching up to do on certain models for vLLM.