Amund Tveit

@0xrodu Recommended relatively recent (2023) paper by Sanjay G. - Towards Modern Development of Cloud Applications - pages.cs.wisc.edu/~rgrandl/paper…

Have often seen Jeff mention Sanjay. Didn't remember reading anything in particular from him or ever coming across an interview from him. Fell into a rabbit hole of searching for content on him and 15 minutes later I still can't find anything. This guy is invisible!

4 bits is all you need (and 3.6 bit you have?) for resource-efficient LLMs? A few months ago OpenAI published their open weights model(s) GPT-OSS (20B and 120B), and one of the eye-catching characteristics was that it was heavily quantized - in other words “shrunk” or compressed - to 4 bits per parameter (MXFP4) instead of the commonly used 16 bit (BF16). 4 bit (“half byte”) means that the model uses much less memory, energy and also can run efficiently on hardware natively supporting 4 bit MXFP inference (e.g. Nvidia Blackwell B200/300 and AMD MI355X). With 4 bit compared to 16 bit there is some quality loss, but typically marginally. But can we go even lower and represent the model in fewer bits per parameter, e.g. 3 bit, 2 bit or even 1 bit? There is highly interesting and seemingly promising research on more efficient quantization (few bit representation) of deep neural networks, however perhaps 4 bit is very close to the lower plateau - based on: a) findings in an article with large empirical studies by Meta, Google DeepMind, Cornell University and Nvidia that language models have a capacity of 3.6 bits per parameter - Language Models Don’t Store Facts: They Compress Patterns - so perhaps 3.6 bit is a lower bound per parameter for efficient representation of deep neural networks? b) and to a lesser degree personal testing on doing further quantizations on GPT-OSS 20B, it seems like when going to 2 and 3 bit even with (LoRA-based) finetuning it is hard to lift it back to close to 4 bit. Amund atsentia.com

AI on Apple Watch

@simonw @NeelNanda5 Re memory trimming / memory in LLm @realSharonZhou Founder of lamini might know more (re docs.lamini.ai/tuning/memory_… )

@simonw Believe that @NeelNanda5 might know more about memory trimming in LLm (it seems close to mechanistic interpretability - neelnanda.io/mechanistic-in… )

What's the latest research on how much baked-in knowledge an LLM needs in order to be useful? If I want a specialist coding model can I trim the size of that model down by stripping out detailed knowledge of human history, geography etc? Can we even do that?

@xamat Hope to get complementary byte-based grokking results scrutinized and published soonish :)

Very cool!

@lateinteraction re interesting papers - perhaps this paper (from 2024) of byte-based models could be interest to try out for DSPy? arxiv.org/abs/2402.19155

@jobergum Congratulations! Curious to learn more :) perhaps 2025 is the year of entrepreneurship..? 👀

New bio dropped today. Let's go!

The *full* Python Documentary will be released this Thursday (Aug 28) at 10am PDT / 19:00 CET. More at discuss.python.org/t/python-docum… Don't miss the online release party / chat! @TECHDOCU

@jobergum yes, quite the jump with reasoning models from O1 and onwards to GPT-5. However, believe many are a bit behind in learning prompting/context engineering skills.

Many made up their mind after trying to code with chatGPT v1 early 2023 and never look back. Even last month there has been significant progress. Better models, better context engineering.

Exciting times in AI (Agent) devtools and vibe coding — and for the historically inclined, you might enjoy a 2008 throwback: my blog post Increase the automation of Test-Driven Development?amundblog.blogspot.com/2008/01/increa…

A new opening for multimodal model research: jobs.careers.microsoft.com/global/en/job/… . Please apply if interested.

Published a paper: Muon Optimizer Accelerates Grokking - arxiv.org/abs/2504.16041 - with colleagues Bjørn Remseth (@rmz) and Arve Skogvold

Grokking transformer implementations in Pytorch and Jax/Flax - amund.blog/pytorch_jax_gr…

A real, practical device technology that provides stable, logical, qubits. We are entering the quantum era. This is not hype.