Andy Hall

1.6K posts

Andy Hall banner
Andy Hall

Andy Hall

@ahall_research

Building free systems. Prof @StanfordGSB, Senior Fellow @HooverInst. Advisor, @a16zcrypto, @ByForumAI. Writing at https://t.co/K0BfKKi4sM

Katılım Ocak 2022
2.5K Takip Edilen11.6K Takipçiler
Sabitlenmiş Tweet
Andy Hall
Andy Hall@ahall_research·
My new research piece: what the politics of jobless prosperity might look like in an AGI world, why the real political backlash to AI hasn’t started yet, and how the labs should prepare. 1. The backlash to AI isn’t here yet. There is anxiety among American voters, but there is no populist backlash yet, because the job losses haven’t started yet—and we don’t even know if they ever will. AI is not in the top 20 issues Americans say they care most about, and the AI policy issue with the most energy right now, data center opposition, reflects not just AI but also NIMBYism, as @mattyglesias has pointed out. 2. Real backlash will happen if and when unemployment climbs by two percentage points, because that’s where data shows we tend to see meaningful electoral effects of unemployment. At that point, if we do not have a good inventory of smart policy ideas ready, we could be overwhelmed with bad ones. 3. The labs should focus more on measurement, and less on dreaming up New Deals. There is tremendous uncertainty about what kind of job displacement there’s going to be. Instead of attempting to write a new social contract from the top down before Americans are even asking for one, the labs should be helping us all get more intel on whether, when, and how job displacement is occurring—building from the helpful data sharing they’ve already started piloting. This will put society in a better position to design policies that make sense for everyone. In doing the research for this piece, I came to two broader realizations. First, there is way more uncertainty than I appreciated about how the economics of AGI might play out, and there is stronger evidence than I appreciated that job losses from AI have not meaningfully started yet. And second, if AGI plays out the way the labs are predicting, the politics will be very hard to forecast, because it will be the politics of “jobless prosperity,” with jobs falling while the economy grows. We have very little experience with this happening at this kind of scale, and it will break our typical models of politics. For both of these reasons, we should all be really humble in making pronouncements about the politics of AGI. I hope my piece will be read in this light, as an attempt to reason about something that is super important but also super hard to forecast accurately. You can check out a lot more in the piece here: freesystems.substack.com/p/the-politics…
Andy Hall tweet media
English
23
35
204
78.4K
Andy Hall retweetledi
MTS
MTS@MTSlive·
.@deredleritt3r on why Kimi K3 is the best open source model but still not close to the frontier: “It is by far the best open source model I’ve tested to date. It is much better than GLM 5.2, which the timeline was in love with two weeks ago.” “I kind of think of this model preliminarily as a very very solid model, but nothing extraordinary when it comes to frontier AI capabilities.” “This model is sensationally good at finding the right authority. So this exact search capability that I just said Fable lacks, this model is the best non-OpenAI model probably that I’ve seen in search.” "But when it comes to analyzing the things that it has found, reasoning logically through a statute or a regulation or an opinion, it's kind of like a random number generator. Sometimes it'll be really smart and really good. Sometimes it'll give me horrendous answers." "It will get some of the easier questions on PrinzBench wrong, where even pretty dumb models you usually get right. But then it'll impress me on some really hard question. It's just a really odd model to test in that regard."
prinz@deredleritt3r

A few thoughts on Kimi K3: - The Moonshot team is absolutely cracked. Some of you might remember that K1.5 was released on January 20, 2025 - i.e., on the same day as DeepSeek R1, and it was ~just as good as R1. IIRC, the model wasn't open-source, which is why no one ever talks about it and DeepSeek got all the glory. - Do K3's capabilities generalize outside of coding, or is it coding-benchmaxxed? In my initial testing, it's materially worse than GPT-5.6 Sol and Fable 5 for certain use cases that do not involve coding. - I will not tire of saying this: the publicly available models in July 2026 *do not matter*. It doesn't matter whether China is 2 months behind the frontier, or 4 months, or 8 months. The only thing that matters is the race to RSI. What do we need for RSI? IMO, two things: (i) a model with excellent research taste; and (ii) tons of compute. Does excellent research taste develop on its own from increasingly strong coding abilities? I'm skeptical. I think it's a much more general ability that is more a function of overall model intelligence than just proficiency at some skill (like coding). - All that said, my estimate for how far behind China is has not changed at all with the release of K3 (IMO, much farther behind than the timeline thinks today). - This model's cyber capabilities are currently unknown, and this is an area worth watching. The few weeks following the release of K3's weights might be interesting times for defenders (or, then again, maybe not). - The geopolitical consequences of the release of this model are also worth watching - both in the U.S. and in China.

English
4
4
32
8.2K
Andy Hall retweetledi
felpix
felpix@felpix_·
topic modeling for congressional press releases republicans really love talking about china!
felpix tweet media
Andy Hall@ahall_research

My new research: I analyzed 280,000 fundraising emails to track the recent, sharp rise in anti-billionaire populist rhetoric among Democratic politicians, and to show how it's slowly merging with a new kind of anti-AI populism. We know from @davidshor, @jasminewsun, @ArchieHall and others' writing and research that American voters are skeptical of AI, but we know less about how politicians at large are thinking about it. Fundraising emails are a super useful way to measure, in roughly real-time, what politicians are saying to their most devoted followers about key issues. Here are some of my main findings: (1) Anti-billionaire rhetoric took off sharply in 2025 among Democrats, driven by anti-Elon fundraising appeals and now including a variety of tech themes. (2) Anti-AI content is only a small fraction of Dem emails even today---but it's rising quickly. (3) Anti-AI Dem emails don't tend to focus on job loss or x-risk; they're focused on how AI is the next thing that billionaires are "doing to us"---the latest symptom of an oligarchy rigging the economy against us. (4) The spike in anti-billionaire populism looks similar to a previous spike in anti-social-media rhetoric among Republicans around 2021. That spike never really turned into meaningful policy. (5) On the other hand, the adoption of the AI topic among Dems is on a similar trajectory to their previous embrace of anti-billionaire rhetoric---so it could be a major focus in the near future. Lots more details in the full write-up here: freesystems.substack.com/p/ai-is-the-de…

English
3
1
16
4.8K
Andy Hall retweetledi
MTS
MTS@MTSlive·
SITUATION EXPLAINED: AI detectors have gotten really good, except at catching AI pretending to write like a specific human author. @EpochAIResearch tested three leading detectors, Pangram, GPTZero, and Originality, on both AI and human text • On plain AI-generated text from basic prompts, false negative rates were near zero, at most 0.7%, detectors are now genuinely good at catching AI, a sharp reversal from a few years ago • Originality had a 4% false positive rate on human writing, Pangram hit 0% false positives and 0% false negatives across 792 tested passages • The gap: all three detectors do noticeably worse when AI is specifically imitating a real human author's writing style rather than just generating from a basic prompt @theojaffee: "This is a total reversal from Claude 2 in 2023, which used to read as 100% human on GPTZero." @schisofrenia: "It must be so annoying to be a professor grading an essay these days, because there's so many subtle ways you can obviously AI your way through it."
Epoch AI@EpochAIResearch

We stress-tested some AI detectors and found that they rarely flag human text as AI-generated. But asking LLMs to mimic a specific author causes detectors to misclassify text as human-generated ~13% of the time. For scientific writing, false negatives rose to ~26%.

English
0
3
28
6.1K
Andy Hall
Andy Hall@ahall_research·
Our new research: to start building towards political superintelligence by using AI to help us govern better, we created an AI that reads contracts and predicts ambiguities that will lead to disputes. We tested it on 10,000 Kalshi and Polymarket resolution rules, and it works quite well! Just by reading the resolution rules, our tool is able to assess which prediction-market contracts are likely to lead to resolution disputes, far better than random chance. So much of governance is writing good rules. But writing good rules is hard. For centuries, we've relied on human experts to try to write rules that anticipate as many confusions, ambiguities, and loopholes as possible. Inevitably, we make mistakes---from the famous contract law example of "the two ships Peerless" that I explore in today's piece, to unclear content moderation policies, ambiguous legislation, or the recent dustup around the US-Iran cease fire agreement wording, this is a perennial challenge. Can AI help us do it better? At Free Systems, a big part of our vision is figuring out how AI can improve how we govern, so we were eager to put this to the test. We collected a sample of 10,000 prediction-market contracts with their stated resolution rules, along with info on which ones ended up disputed. Then, we worked with our buddy Claude to develop a 10-point rubric for contract clarity, covering elements like whether the key question is well defined, whether the entities are identifiable, whether the time window is clearly specified, and so on. We had an LLM grader apply the rubric to the contracts, then built a simple machine-learning model that uses the 10-dimensional rubric score to predict subsequent disputes. The resulting scores allow us to provide overall grades to prediction-market contracts which reflect how clearly written they are and how likely they are to fall into dispute later. The contracts we grade "CCC" are 3.4x more likely to fall into dispute than the ones we grade "A." There's a lot more work to do here---we need to make sure our predictions hold in a truly out-of-sample test where we grade contracts now and see if they get disputed in the future, which we'll be working on next---and we need to expand this beyond prediction market contracts as well. But we're super excited about this direction. Tools like this will help us to identify contract ambiguities before they become disputes, allowing us to write better rules, improve governance, and eventually, get to political superintelligence. There's lots more info in our write-up, here: freesystems.substack.com/p/superintelli… Joint work with @elliotjpaschal
Andy Hall tweet media
English
14
32
175
30.3K
Andy Hall retweetledi
jasmine sun
jasmine sun@jasminewsun·
data center road trip // day 2-3 - Port Washington has faced intense protests and a mayoral recall over their data center. residents cite environment and aesthetics, but are fine with the power plant in the middle of town - Mount Pleasant officials were relieved to have Microsoft build, filling the $100m+ debt hole left by the failed Foxconn project - the union guys are making bank, getting $100 cash per diems on top of 6-figure salaries, and farmers became decamillionaires selling their land - the huge numbers on data center deals make residents *more* suspicious, not less! I’ve heard less about AI than NDAs and council meetings: people are furious at local gov for “selling out,” giving tax subsidies, and blindsiding them - I’ve heard “I’m not a conspiracy guy but…” at least 4x. everyone thinks everyone else is being paid off
jasmine sun tweet mediajasmine sun tweet mediajasmine sun tweet mediajasmine sun tweet media
jasmine sun@jasminewsun

I’m spending the next week driving around Wisconsin and Michigan to get a firsthand view of the data center buildout and backlash am visiting sites, talking to tons of people — tradespeople, activists, planners, residents — then writing up a story lmk what questions I should dig into!

English
7
6
118
12.5K
Andy Hall retweetledi
Olivia Moore
Olivia Moore@omooretweets·
Only 2.2% of U.S. households have a paid AI subscription We are still so early
Olivia Moore tweet media
English
206
221
2.7K
315.2K
Ryan Greenblatt
Ryan Greenblatt@RyanGreenblatt·
I did some quick tests that indicated that the Kimi K3 pretrain is around halfway between Opus 4 and Opus 4.5. So ~10 months behind Anthropic. These tests probably understate data improvements, so overall I think it's a similarly good pretrain to Opus 4.5 (~8 months behind). These tests are better at measuring "general pretrain capability" than at incorporating (coding-specific) data quality. This was prompted by me thinking more and realizing that my claim that "As a pretrain, it's probably somewhere between 4.8 and Mythos (around halfway between?)" was probably too bullish on the model and that I might as well test and find out. (And yep, this was very wrong.) I think Mythos is a pretty big step up in pretraining, so K3 might be more than 8 months behind on the historical pretraining trend relative to Mythos (as in, Mythos is >>3 months ahead of K3 and Mythos was fully done training ~5 months ago). Overall, this makes me suspect more of the improvements are due to distillation-type effects and makes me think the full catch-up times would be somewhat longer (if Ant/OpenAI stopped but investment still followed current trends). Minimally, more of the improvement probably lives in post-training/mid-training. For reference, the same test indicates K2.6 is around halfway between Sonnet 4 and Sonnet 4.5. (And this roughly corresponds to some other similar measures.) Sorry about the error.
Ryan Greenblatt@RyanGreenblatt

Kimi K3 was significantly but not massively above my expectations. I'd tentatively guess it's similar in overall usefulness/usability to Opus 4.8 and in overall capability somewhat above Opus 4.8 (while also being somewhat more benchmaxxed). As a pretrain, it's probably somewhere between 4.8 and Mythos (around halfway between?). Maybe this implies Kimi is like 8 or so months behind Anthropic in overall model strength/goodness (including usability) and like 6 or so months behind on overall capability (somewhat below Mythos Preview). This gap is presumably reduced by distillation (and more generally using OpenAI/Anthropic models) and algorithm leakage/diffusion, so I think that hypothetically if the US completely stopped and recent algos didn't diffuse, it would maybe take Kimi like 10 months to fully catch up to the best internal (including in development) Anthropic model. (I think this notion might be a better measure of where Anthropic/OpenAI are relative to Kimi, even though this hypothetical won't happen.) And if the US completely stopped, it might take Kimi around 27 months to reach the level the US would otherwise have reached one year from now (as in, with a year of further progress). My views here are pretty sensitive to how much benchmark performance is representative to overall usability. I think I now expect an open-weight AI which is straightforwardly "Mythos-level at cyber" (including usability etc.) in like 5 months supposing Kimi and others don't change their open-weight model policy. (I don't have a strong view about how big of a deal this is for cyber, but it may cause significant political consequences. This could be a significant overestimate of the time required.) I wonder what's driving Kimi being closer than I would have expected. Options include: - Experiment compute is significantly less important than labor (and labor at Kimi is competitive, which seems super plausible) - Implies more of a speedup from AI automating AI R&D and a bigger software-only intelligence explosion. - Or possibly Kimi is just doing much better than US companies and this is overcoming experiment compute disadvantages. - Algorithms are diffusing a lot / quickly (from e.g. OpenAI to Kimi). - Perf is overstated / benchmaxxed a lot. - Distillation / using OpenAI or Anthropic frontier AIs in AI development is very helpful for catching up. (But I'd guess Kimi K3 is a competitive pretrain which distillation doesn't help with?) - US companies aren't going as fast as they could for whatever reason.

English
25
36
444
76.9K
Andy Hall
Andy Hall@ahall_research·
@RyanGreenblatt Appreciate your work and your transparency! It’s incredibly valuable
English
0
0
9
616
Andy Hall retweetledi
MTS
MTS@MTSlive·
Neurosurgeon Jonathan Slotkin reveals the interest group he says is quietly running the biggest opposition to autonomous vehicles: "Plaintiffs' bar is in significant opposition to many aspects of the development of this technology, and has been formally as early as 2017 and 2018 through their lobbying group." "I can tell you some facts. The majority of state liability actions that make it to courtrooms are automobile litigation. I believe it's something like 60%. This is a very significant driver of their business." "If I'm them, this is me speculating, staring down the prospect of 90% decreases in serious injuries. Their charitable view is that they want to make sure that these companies like Waymo, Zoox, and Tesla can still be subject to liability." "If you told me I had to be injured tomorrow and go to court and try to get money from Waymo or go to court and try to get money from an under-insured motorist that's unemployed, I'll take the first one." "That is probably one of the bigger drivers for opposition. And in my opinion, that's part of why it tends to be blue areas or bleeders of blue areas that are most opposed." @slotkinjr
English
2
7
41
8.7K
Andy Hall
Andy Hall@ahall_research·
Across prediction markets, politics and similar topics seem to have the highest rates of resolution disputes.
Andy Hall tweet media
Andy Hall@ahall_research

Our new research: to start building towards political superintelligence by using AI to help us govern better, we created an AI that reads contracts and predicts ambiguities that will lead to disputes. We tested it on 10,000 Kalshi and Polymarket resolution rules, and it works quite well! Just by reading the resolution rules, our tool is able to assess which prediction-market contracts are likely to lead to resolution disputes, far better than random chance. So much of governance is writing good rules. But writing good rules is hard. For centuries, we've relied on human experts to try to write rules that anticipate as many confusions, ambiguities, and loopholes as possible. Inevitably, we make mistakes---from the famous contract law example of "the two ships Peerless" that I explore in today's piece, to unclear content moderation policies, ambiguous legislation, or the recent dustup around the US-Iran cease fire agreement wording, this is a perennial challenge. Can AI help us do it better? At Free Systems, a big part of our vision is figuring out how AI can improve how we govern, so we were eager to put this to the test. We collected a sample of 10,000 prediction-market contracts with their stated resolution rules, along with info on which ones ended up disputed. Then, we worked with our buddy Claude to develop a 10-point rubric for contract clarity, covering elements like whether the key question is well defined, whether the entities are identifiable, whether the time window is clearly specified, and so on. We had an LLM grader apply the rubric to the contracts, then built a simple machine-learning model that uses the 10-dimensional rubric score to predict subsequent disputes. The resulting scores allow us to provide overall grades to prediction-market contracts which reflect how clearly written they are and how likely they are to fall into dispute later. The contracts we grade "CCC" are 3.4x more likely to fall into dispute than the ones we grade "A." There's a lot more work to do here---we need to make sure our predictions hold in a truly out-of-sample test where we grade contracts now and see if they get disputed in the future, which we'll be working on next---and we need to expand this beyond prediction market contracts as well. But we're super excited about this direction. Tools like this will help us to identify contract ambiguities before they become disputes, allowing us to write better rules, improve governance, and eventually, get to political superintelligence. There's lots more info in our write-up, here: freesystems.substack.com/p/superintelli… Joint work with @elliotjpaschal

English
3
3
27
3.5K
Andy Hall
Andy Hall@ahall_research·
We find that three key flaws are the strongest predictors of prediction market contract disputes: (1) Core question vaguely defined (2) Unclear which entity "counts" for the condition to be met (3) The settlement source is not identified specifically Other flaws that our LLM grader finds turn out to be less relevant. Of course, the fact that these issues predict disputes does not inherently mean that correcting them in the initial contract would prevent the dispute -- it might be that these issues tend to crop up in contracts that are inherently harder to resolve.
Andy Hall tweet media
Andy Hall@ahall_research

Our new research: to start building towards political superintelligence by using AI to help us govern better, we created an AI that reads contracts and predicts ambiguities that will lead to disputes. We tested it on 10,000 Kalshi and Polymarket resolution rules, and it works quite well! Just by reading the resolution rules, our tool is able to assess which prediction-market contracts are likely to lead to resolution disputes, far better than random chance. So much of governance is writing good rules. But writing good rules is hard. For centuries, we've relied on human experts to try to write rules that anticipate as many confusions, ambiguities, and loopholes as possible. Inevitably, we make mistakes---from the famous contract law example of "the two ships Peerless" that I explore in today's piece, to unclear content moderation policies, ambiguous legislation, or the recent dustup around the US-Iran cease fire agreement wording, this is a perennial challenge. Can AI help us do it better? At Free Systems, a big part of our vision is figuring out how AI can improve how we govern, so we were eager to put this to the test. We collected a sample of 10,000 prediction-market contracts with their stated resolution rules, along with info on which ones ended up disputed. Then, we worked with our buddy Claude to develop a 10-point rubric for contract clarity, covering elements like whether the key question is well defined, whether the entities are identifiable, whether the time window is clearly specified, and so on. We had an LLM grader apply the rubric to the contracts, then built a simple machine-learning model that uses the 10-dimensional rubric score to predict subsequent disputes. The resulting scores allow us to provide overall grades to prediction-market contracts which reflect how clearly written they are and how likely they are to fall into dispute later. The contracts we grade "CCC" are 3.4x more likely to fall into dispute than the ones we grade "A." There's a lot more work to do here---we need to make sure our predictions hold in a truly out-of-sample test where we grade contracts now and see if they get disputed in the future, which we'll be working on next---and we need to expand this beyond prediction market contracts as well. But we're super excited about this direction. Tools like this will help us to identify contract ambiguities before they become disputes, allowing us to write better rules, improve governance, and eventually, get to political superintelligence. There's lots more info in our write-up, here: freesystems.substack.com/p/superintelli… Joint work with @elliotjpaschal

English
0
2
21
2.2K
Andy Hall
Andy Hall@ahall_research·
@DarenMatsuoka Oh man I hadn’t heard of that one — just had grok summarize. Did the contract really just say the weather would be “good”? lol
English
1
0
0
279
Max Kagan
Max Kagan@max_kagan·
@ahall_research Definitely. But I think language can only go so far to resolve the complexity of the real world vs. redefining the real world to needs of markets. I doubt any amount of foresight can make a "Is the strait open?" market function as well as "price of a bbl of Dubai crude"
English
1
0
0
164
Andy Hall
Andy Hall@ahall_research·
@max_kagan I think you need both. There are definitely sharp limits to how clear you can make a contract. But there is low hanging fruit too
English
1
0
0
210
Max Kagan
Max Kagan@max_kagan·
@ahall_research Not sure if you've read Cronon's Nature Metropolis (one of my fav books), but he makes case that financial markets matured not by writing better/clearer rules to match the complexity of the real world but by reducing the complexity of the real world to match the needs of markets
English
1
0
1
273