Peter Wang 🦋

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.

🚨: Physicists have discovered that time can indeed behave as if it is moving in two directions at once at the quantum scale.

This is precisely the reason that I have been so supportive of open-source over the years. It is an insurance policy against both tyrannical governments and the death throes of the industrial-era nation state, the latter of which I fear we are living through. Absent the oh-so-careful navigation of the narrow path (which I tried), it seemed inevitable that closed-source models would be captured by USG and thus subject to all of USG’s many mood swings, and thus intrinsically unreliable for both businesses and other countries. Alas, I did what I could to help us walk the narrow path, as did countless others. But the odds were always stacked against us. The hybrid open/closed model world was a good future. It would mean that AI is an intrinsically profitable business, favoring Western and especially American capitalistic institutions. Yet the power of the closed labs would be checked by open-weight models that, while behind the cutting-edge, provided balance. This supply chain risk designation will render closed-source models less attractive to many customers worldwide. The regulatory risk is now extremely high. USG cannot unfire the gun that it has fired—or if it can, doing so would take an extreme measure of skill, determination, and sensitivity that I do not think will be forthcoming. Thus the balanced open/closed world seems less likely on the margin, and instead it is likelier that we live in the open-source-dominant world now. This world probably favors China, because they have a political economy more favorable to large-scale subsidy of what is essentially digital public infrastructure. It also probably favors Chinese-style digital and physical surveillance (but made all the more pervasive and capable with advanced AI), since the catastrophic misuse risks of open source are higher. Chinese-style institutions have an overall structural advantage in this future, it seems to me, and Western institutions have a structural disadvantage. You can argue this has always been true, but USG just increased the likelihood of AI futures where the US is at an inherent disadvantage. And this is to say nothing of the damage to the business environment, the AI industry, etc., about which I have spoken earlier. The future is likelier to be more decentralized, more confusing, and more dangerous than it was a few weeks ago. Yet it also may be brighter; it is probably a higher variance future than the steadier transition of the narrow path. Perhaps you like that trade off. Nonetheless, we have probably been knocked off the narrow path, and the odds of a “normal” transition to the era of machine intelligence are now meaningfully lower. Humans are living through moving history once more.