Max Kirkby

"There are even LLMs submitting proofs for Fermat's last theorem". My brother you are having this conversation in Oxford, Andrew Wiles lives within a 2 mile radius.

this is genuinely how it feels to work on the (Australian) post-training team at @baseten

As a researcher it's pretty cool to work hand in hand with this kind of engineering talent. Fewer bugs wins, and knowing every layer of every stack is pretty critical in making that happen.

@matt_slotnick the secret is friendship

@rapprach Speed and perfmogging

@DanielCHTan97 Thank you, just the start of a bigger agenda :)

Cool research! Tl;dr consider trying to prompt-distill a constitution into a model. What turns out to matter is (i) on-policy learning (sample from the student, don't use off-policy demonstrations) and (ii) dense feedback (on-policy distillation, not RL)

@mj_bilodeau My intuition is that OPSD should stabilise constitutional behaviour, especially with eg an EMA teacher. I do feel like one neat next test is repeated reruns on new diverse datasets to see which principles might upweight/downweight/drift

@maxkirkby @maxkirkby any thoughts or intuition on whether the on-policy dense methods make constitutional behavior more stable or more prone to drift over repeated post-training runs?

@stefanopopoulos This one worked out well

New spread trade just dropped: - Sell linear attention and DSA - Buy KV-cache compaction

@amiruci Everything is world

You’ve used language models, image models, video models, and voice models. Now it’s time for world models, thanks to World Labs.

@philipkiely Time to get back into reading books 💚

Inference Engineering launches today. baseten.com/inference-engi…

There’s a funny latent divide between configuring a training run and the person next to you writing an essay on whether AI should be allowed to exist.

Come join @baseten where you get to research and be part of an index on AI to expose yourself to all of it

This was fun. We replicated the results of black box on-policy distillation on the @baseten stack, underscoring the potential of different architectures for post-training!

Continuing this week with a case study ☕️ How did @sullyai return 30M+ clinical minutes to doctors? By ditching closed-source models for a high-performance open-source stack on Baseten. Like many companies, Sully faced inference challenges as they scaled, with ballooning proprietary model costs and unpredictable latency. This was especially critical in Sully's case: in a live clinical setting, a 70-second wait is an eternity. To solve this challenge, we worked together to move to open-source models like GPT OSS 120b. With the Baseten inference stack, Sully was live on NVIDIA HGX B200s just 2 days after the model's release. The results: • 90% reduction in inference costs • 65% reduction in median latency • 21x Return on Agent Spend (ROAS) Check out the full story: baseten.co/resources/cust…

Mathematics and mathematicians get everywhere. Like, everywhere. Andrew Wiles @OxUniMaths sampled by rapper J Cole. Fermat's latest theorem.

LLMs are amnesiacs. Once context fills up, they forget everything. To fight this means grappling with a core question: how do you update a neural network without breaking what it already knows? In this piece, @oneill_c and @part_harry_ argue that continual learning is inseparable from specialization. While there are various ideas to allow generalist models to learn everything without forgetting anything, these ideas are fundamentally in tension with continual learning in general. What comes after monolith models? A Cambrian explosion of specialists. Read more here: baseten.co/resources/rese…

He writes well, doesn't he