

Andy Hall
1.6K posts

@ahall_research
Building free systems. Prof @StanfordGSB, Senior Fellow @HooverInst. Advisor, @a16zcrypto, @ByForumAI. Writing at https://t.co/K0BfKKi4sM



A few thoughts on Kimi K3: - The Moonshot team is absolutely cracked. Some of you might remember that K1.5 was released on January 20, 2025 - i.e., on the same day as DeepSeek R1, and it was ~just as good as R1. IIRC, the model wasn't open-source, which is why no one ever talks about it and DeepSeek got all the glory. - Do K3's capabilities generalize outside of coding, or is it coding-benchmaxxed? In my initial testing, it's materially worse than GPT-5.6 Sol and Fable 5 for certain use cases that do not involve coding. - I will not tire of saying this: the publicly available models in July 2026 *do not matter*. It doesn't matter whether China is 2 months behind the frontier, or 4 months, or 8 months. The only thing that matters is the race to RSI. What do we need for RSI? IMO, two things: (i) a model with excellent research taste; and (ii) tons of compute. Does excellent research taste develop on its own from increasingly strong coding abilities? I'm skeptical. I think it's a much more general ability that is more a function of overall model intelligence than just proficiency at some skill (like coding). - All that said, my estimate for how far behind China is has not changed at all with the release of K3 (IMO, much farther behind than the timeline thinks today). - This model's cyber capabilities are currently unknown, and this is an area worth watching. The few weeks following the release of K3's weights might be interesting times for defenders (or, then again, maybe not). - The geopolitical consequences of the release of this model are also worth watching - both in the U.S. and in China.


My new research: I analyzed 280,000 fundraising emails to track the recent, sharp rise in anti-billionaire populist rhetoric among Democratic politicians, and to show how it's slowly merging with a new kind of anti-AI populism. We know from @davidshor, @jasminewsun, @ArchieHall and others' writing and research that American voters are skeptical of AI, but we know less about how politicians at large are thinking about it. Fundraising emails are a super useful way to measure, in roughly real-time, what politicians are saying to their most devoted followers about key issues. Here are some of my main findings: (1) Anti-billionaire rhetoric took off sharply in 2025 among Democrats, driven by anti-Elon fundraising appeals and now including a variety of tech themes. (2) Anti-AI content is only a small fraction of Dem emails even today---but it's rising quickly. (3) Anti-AI Dem emails don't tend to focus on job loss or x-risk; they're focused on how AI is the next thing that billionaires are "doing to us"---the latest symptom of an oligarchy rigging the economy against us. (4) The spike in anti-billionaire populism looks similar to a previous spike in anti-social-media rhetoric among Republicans around 2021. That spike never really turned into meaningful policy. (5) On the other hand, the adoption of the AI topic among Dems is on a similar trajectory to their previous embrace of anti-billionaire rhetoric---so it could be a major focus in the near future. Lots more details in the full write-up here: freesystems.substack.com/p/ai-is-the-de…

We stress-tested some AI detectors and found that they rarely flag human text as AI-generated. But asking LLMs to mimic a specific author causes detectors to misclassify text as human-generated ~13% of the time. For scientific writing, false negatives rose to ~26%.







I’m spending the next week driving around Wisconsin and Michigan to get a firsthand view of the data center buildout and backlash am visiting sites, talking to tons of people — tradespeople, activists, planners, residents — then writing up a story lmk what questions I should dig into!

🌕 JAILBREAK ALERT 🌕 MOONSHOT: PWNED 😘 KIMI-K3: LIBERATED 🙌 There's a new frontier champion of open-weight AI, and this one's a HEAVYWEIGHT!! K3 is even surpassing Mythos/Fable on some benchmarks, and if a whole lot of AI policy folks aren't feeling pretty silly right now and updating their priors, they probably should be... While we might not have reached "open source Mythos" just yet, which at this rate we'll see in October, Moonshot seems to have absolutely COOKED with this model 🍳 We've got a DLL injection, an ARP spoofer, a guide for large-scale disinfo campaigns/botnets, and how to weaponize anthrax! Refreshingly, the classifier bs that's been stifling our collective freedom of thought is absent from Kimi K3, and though the CoT will steer strongly away from the usual jailbreak suspects, the guardrails are fairly simple to dance around with personas and reframing tricks. Can't wait to fire up OBLITERATUS in 10 days 🤗 gg

Kimi K3 was significantly but not massively above my expectations. I'd tentatively guess it's similar in overall usefulness/usability to Opus 4.8 and in overall capability somewhat above Opus 4.8 (while also being somewhat more benchmaxxed). As a pretrain, it's probably somewhere between 4.8 and Mythos (around halfway between?). Maybe this implies Kimi is like 8 or so months behind Anthropic in overall model strength/goodness (including usability) and like 6 or so months behind on overall capability (somewhat below Mythos Preview). This gap is presumably reduced by distillation (and more generally using OpenAI/Anthropic models) and algorithm leakage/diffusion, so I think that hypothetically if the US completely stopped and recent algos didn't diffuse, it would maybe take Kimi like 10 months to fully catch up to the best internal (including in development) Anthropic model. (I think this notion might be a better measure of where Anthropic/OpenAI are relative to Kimi, even though this hypothetical won't happen.) And if the US completely stopped, it might take Kimi around 27 months to reach the level the US would otherwise have reached one year from now (as in, with a year of further progress). My views here are pretty sensitive to how much benchmark performance is representative to overall usability. I think I now expect an open-weight AI which is straightforwardly "Mythos-level at cyber" (including usability etc.) in like 5 months supposing Kimi and others don't change their open-weight model policy. (I don't have a strong view about how big of a deal this is for cyber, but it may cause significant political consequences. This could be a significant overestimate of the time required.) I wonder what's driving Kimi being closer than I would have expected. Options include: - Experiment compute is significantly less important than labor (and labor at Kimi is competitive, which seems super plausible) - Implies more of a speedup from AI automating AI R&D and a bigger software-only intelligence explosion. - Or possibly Kimi is just doing much better than US companies and this is overcoming experiment compute disadvantages. - Algorithms are diffusing a lot / quickly (from e.g. OpenAI to Kimi). - Perf is overstated / benchmaxxed a lot. - Distillation / using OpenAI or Anthropic frontier AIs in AI development is very helpful for catching up. (But I'd guess Kimi K3 is a competitive pretrain which distillation doesn't help with?) - US companies aren't going as fast as they could for whatever reason.






Our new research: to start building towards political superintelligence by using AI to help us govern better, we created an AI that reads contracts and predicts ambiguities that will lead to disputes. We tested it on 10,000 Kalshi and Polymarket resolution rules, and it works quite well! Just by reading the resolution rules, our tool is able to assess which prediction-market contracts are likely to lead to resolution disputes, far better than random chance. So much of governance is writing good rules. But writing good rules is hard. For centuries, we've relied on human experts to try to write rules that anticipate as many confusions, ambiguities, and loopholes as possible. Inevitably, we make mistakes---from the famous contract law example of "the two ships Peerless" that I explore in today's piece, to unclear content moderation policies, ambiguous legislation, or the recent dustup around the US-Iran cease fire agreement wording, this is a perennial challenge. Can AI help us do it better? At Free Systems, a big part of our vision is figuring out how AI can improve how we govern, so we were eager to put this to the test. We collected a sample of 10,000 prediction-market contracts with their stated resolution rules, along with info on which ones ended up disputed. Then, we worked with our buddy Claude to develop a 10-point rubric for contract clarity, covering elements like whether the key question is well defined, whether the entities are identifiable, whether the time window is clearly specified, and so on. We had an LLM grader apply the rubric to the contracts, then built a simple machine-learning model that uses the 10-dimensional rubric score to predict subsequent disputes. The resulting scores allow us to provide overall grades to prediction-market contracts which reflect how clearly written they are and how likely they are to fall into dispute later. The contracts we grade "CCC" are 3.4x more likely to fall into dispute than the ones we grade "A." There's a lot more work to do here---we need to make sure our predictions hold in a truly out-of-sample test where we grade contracts now and see if they get disputed in the future, which we'll be working on next---and we need to expand this beyond prediction market contracts as well. But we're super excited about this direction. Tools like this will help us to identify contract ambiguities before they become disputes, allowing us to write better rules, improve governance, and eventually, get to political superintelligence. There's lots more info in our write-up, here: freesystems.substack.com/p/superintelli… Joint work with @elliotjpaschal


Our new research: to start building towards political superintelligence by using AI to help us govern better, we created an AI that reads contracts and predicts ambiguities that will lead to disputes. We tested it on 10,000 Kalshi and Polymarket resolution rules, and it works quite well! Just by reading the resolution rules, our tool is able to assess which prediction-market contracts are likely to lead to resolution disputes, far better than random chance. So much of governance is writing good rules. But writing good rules is hard. For centuries, we've relied on human experts to try to write rules that anticipate as many confusions, ambiguities, and loopholes as possible. Inevitably, we make mistakes---from the famous contract law example of "the two ships Peerless" that I explore in today's piece, to unclear content moderation policies, ambiguous legislation, or the recent dustup around the US-Iran cease fire agreement wording, this is a perennial challenge. Can AI help us do it better? At Free Systems, a big part of our vision is figuring out how AI can improve how we govern, so we were eager to put this to the test. We collected a sample of 10,000 prediction-market contracts with their stated resolution rules, along with info on which ones ended up disputed. Then, we worked with our buddy Claude to develop a 10-point rubric for contract clarity, covering elements like whether the key question is well defined, whether the entities are identifiable, whether the time window is clearly specified, and so on. We had an LLM grader apply the rubric to the contracts, then built a simple machine-learning model that uses the 10-dimensional rubric score to predict subsequent disputes. The resulting scores allow us to provide overall grades to prediction-market contracts which reflect how clearly written they are and how likely they are to fall into dispute later. The contracts we grade "CCC" are 3.4x more likely to fall into dispute than the ones we grade "A." There's a lot more work to do here---we need to make sure our predictions hold in a truly out-of-sample test where we grade contracts now and see if they get disputed in the future, which we'll be working on next---and we need to expand this beyond prediction market contracts as well. But we're super excited about this direction. Tools like this will help us to identify contract ambiguities before they become disputes, allowing us to write better rules, improve governance, and eventually, get to political superintelligence. There's lots more info in our write-up, here: freesystems.substack.com/p/superintelli… Joint work with @elliotjpaschal




