Trevor Levin

At its current exponential growth, Anthropic's annualized revenue will hit 100% of global GDP in early 2028. Do I think this will happen? No. Is it insane that this is the current trajectory, and we should all be preparing for AI to rapidly change the world we live in? Yes.

Of the 228 tasks in our suite, only 5 are estimated as 16+ hours long, making measurements at this range unstable and less meaningful than at ranges with better task coverage. Thus, we are not highlighting exact estimates for models above 16 hours measured with our current suite.

We evaluated an early version of Claude Mythos Preview for risk assessment during a limited window in March 2026. We estimated a 50%-time-horizon of at least 16hrs (95% CI 8.5hrs to 55hrs) on our task suite, at the upper end of what we can measure without new tasks.

We started by investigating why Claude chose to blackmail. We believe the original source of the behavior was internet text that portrays AI as evil and interested in self-preservation. Our post-training at the time wasn’t making it worse—but it also wasn’t making it better.

This one surprised me. Teen girls now say "having lots of money" is important in life at the same rate as boys. There used to be a huge gap.

Yoshua Bengio thinks he knows how to make provably safe superintelligent agents. Bengio built the foundations of modern AI and is the most cited living scientist. He believes his alternative training setup would: 1. Guarantee honesty 2. Prevent unintended goals 3. Produce capable agents 4. Port over most data and techniques from current LLMs 5. Not be inherently more expensive, and perhaps be more intelligent Bengio claims the honesty and lack of unintended goals can be proven mathematically, at least given particular assumptions. And his new organization, LawZero, is aiming to build a scrappy prototype as soon as possible. The architecture is called 'Scientist AI' and it's based on training a model to explain empirical observations, including what people say, rather than training AIs that mimic human behaviour or seek our approval. (Bengio's frank assessment is that "reinforcement learning is evil" and that allowing AIs to independently train their successors is "the most crazy, dangerous bet that unfortunately we are on track to do.") But skeptics question whether Scientist AI really does solve the fundamental problem of 'eliciting latent knowledge' from AI models. And with the commercial race for superintelligence so intense, it's not clear whether the proposal will be able to compete or have time to bear fruit, even if it's sound in theory. On The 80,000 Hours Podcast, links below – enjoy! • Making AI honest and safe (00:00:00) • Scientist AI in plain English (00:02:27) • How Scientist AI differs from LLMs (00:06:32) • How the training data works (00:14:02) • Can this become an agent? (00:21:02) • Why Yoshua is now more optimistic (00:32:11) • Why companies can’t stop racing (00:36:35) • A working prototype won't take long (00:49:15) • Scientist models might be more capable (00:53:34) • “Reinforcement learning is evil” (01:01:27) • Scientist AI from guardrail to agent (01:08:37) • Can safe AI still be competent? (01:12:38) • How much will this cost? (01:19:29) • Can it generalise beyond maths and science? (01:23:26) • A multi-national push for superintelligence (01:39:19) • Want to work with or fund Yoshua? (01:51:16) • Why smart people ignore AI risk (01:54:45) • Don’t let AI build the next AI (02:01:33) • Why politicians miss the real risks (02:12:28) • Why Yoshua changed his mind about AI risk (02:21:27)

Two new roles just opened in the California government to help implement SB 53, the nation's first frontier AI law! Both are in the Department of Technology (CDT), which recommends changes to SB 53's key definitions. 1️⃣ Emerging Technology Program Manager 2️⃣ AI Policy Fellow

Thanks for writing this! And in particular for engaging carefully with my own thoughts on the question. For my part, to avoid any confusion--I don’t think you’ve had this confusion, but some people have--my view on the horse analogy isn’t that we can *deduce* that the labor share will go to zero from what’s happened to the horse share, just that any good argument for the labor share staying non-negligible has to be one that wouldn’t have applied to horses as well. I took issue with Alex Imas’s piece for not passing that test: the pieces of evidence used to support the persistent high labor share view (e.g. that relational goods are luxuries in the cross-sectional sense) would also have applied to horses, but of course technological change in the form of new goods (better means of plowing and transportation) was enough to outweigh the analogous effects that would otherwise have supported the horse share. But again, as you emphasize, that’s not to say they won’t be enough in the human case. As for which effects will predominate in the human case, I agree that the Comin et al. evidence on our shift from manufacturing to services over the last generation or two--due mainly to a kind of income effect--tells us something. But I don’t find it as informative as you do, for two reasons. -- First, there’s the Hubmer point that even over this timeframe (at least in the US), the rising capital content of both manufacturing and services has outweighed the shift to services, “Simpson’s paradox”-style; so the labor share has fallen a little. This is essentially the sort of dynamic I’m expecting to play out in the long run. -- Second and much more importantly, we haven’t yet crossed the threshold of having machines that can do everything much better than we can except share our human identity, so any such extrapolations might be like trying to learn about the 20th-century horse share from 16th-century technological advances.* This is why it seems more informative to me to look at other cases of the share of good X following the introduction of an expanding range of goods Y which are superior on every dimension except some people’s brand loyalty / nostalgia / attachment / relationship to X. Maybe a way to put it is: we’re trying to learn about what happens when “Y beats humans in every way except nostalgia/etc”. Presumably we agree that the reference classes of (1) “Y beat X in every way except nostalgia/etc. but X was not humanity” and (2) “Y only partially beat X on the non-nostalgia fronts but X was humanity” should both be informative. I’d just say it seems to me that people are usually biased toward thinking of humans as categorically different from other sorts of useful systems, i.e. toward giving (2) too much weight relative to (1). [*Why assume we will have such machines one day at all? It’s feasible in principle to build robots that are physically and cognitively indistinguishable from humans (say, unless we cut them up or scrutinize their skin under a microscope). If the GDP share of *non*- human-intrinsic labor stays bounded above zero and growth continues, the payoff to building such a machine will grow indefinitely.] Finally, I just want to flag that what first got me into this line of business--arguing that the labor share will ultimately decline indefinitely--was a thought about the future of economic inequality, and maybe the distribution of power downstream of it, not absolute labor incomes. In fact among my friends, I’m usually the crazy one for arguing that (contra the horse analogy) our wages could stay high or rise, with the labor share declining more slowly than the pie is growing. On the inequality front, there’s less of a discontinuity around whether the labor share asymptotes to 0% rather than your 4.5% example... it seems to me that the forces that could drive it down to 4.5% could probably keep the trend going indefinitely, but that's not a hill I'll die on.

We're hiring grantmakers and senior generalists across our Global Catastrophic Risks teams. Right now, our biggest constraint is people, not funding, which means every strong hire directly translates into more critical work getting done. 🧵

Alex Bores officially takes the lead on Kalshi! It's a relatively shallow market, so one or two big bets can cause some serious fluctuation. And I would guess Bores polls especially well with Kalshi users (most of whom are probably not in-district). But still...

"negotiate a deal with Xi Jinping to impose identical regulation and pre-release vetting on open-weight models in China." Good luck with that

We're hiring Machine Learning Scientists for our election forecasting team! Our election forecasting team works with the largest collection of political data ever assembled and helps allocate billions of dollars in resources - it's a pretty cool job - come work with us!

We're hiring grantmakers and senior generalists across our Global Catastrophic Risks teams. Right now, our biggest constraint is people, not funding, which means every strong hire directly translates into more critical work getting done. 🧵