Ted Sanders

@nabla_theta @MatthewJBar One big corporation I used to work for is Netflix. When I was there, they proactively cancelled subscriptions that had no streaming activity. This cost them millions and I don’t think they got much public goodwill for it. It felt ethical and I’m pleased they did it.

@nabla_theta @MatthewJBar In contrast, I recently cancelled a haircut with a sole proprietor because I was sick. He chose to charge me the full amount anyway because I cancelled less than 24h ahead of time.

new post: Corporations seem evil because we anthropomorphize them if you model companies as people, then they would be amoral sociopaths. but so would your lawnmower. to change corporate behavior effectively, you have to treat them as optimizers and change their incentives.

@sriramk @alexolegimas my understanding is that the record stagnated due to WWII more than any psychological barriers: scienceofrunning.com/2017/05/t

model capabilities and the "four minute mile": been discussing with some frontier lab researcher friends as to why the frontier model capabilities are always so clustered together as opposed to any one model having an unassailable edge. the best metaphor for this in my mind is the "four minute mile": no one broke it till Bannister in 1954 and then very quickly five more runners did it in the next two years. In the model world, this translates to a) intense competitive pressure. b) often similar pool of ideas / research directions c) roughly similar access to capitalization and compute infrastructure the oft quoted example is after the launch of o1 being quickly followed by reasoning models from multiple players both closed and open weights. this is not the case with many other technology driven industries where capability or advancements often tend to be longer held.

@NateSilver538 @SpecialPuppy1 feels very probable to me that in ten years AI becomes as indispensable to people as phones and the internet. in contrast, feels much less probable that search engines and video streaming services are as popular as today.

@SpecialPuppy1 What if there's a 1% chance they're worth basically infinity dollars b/c the singularity hypothesis is true and they win the race. But also even ignoring that Facebook has double the market cap and OpenAI seems like a much better bet even with the 50% haircut.

Reminder that OpenAI's current $850 billion valuation assumes that it will be the most profitable company in the world in just the next 10 years

@simonw @vkawadia yeah, prompt injection is definitely not solved. we're way, way better than couple years ago, but it's very hard to be perfect against such a high dimensional distribution. way more possible prompts than atoms in the observable universe...

@vkawadia "Solved" implies that it's not possible to craft an attack that works. I'm not convinced that's true, even as attacks get harder The big challenge is proving you can't get an attack through - I'm not sure how you'd do that

Hot take: latest reasoning models have solved prompt injection. To demo an attack now, you have to deliberately prompt the model to be gullible. In real-world use, it just doesn't happen. @simonw, still concerned about it?

@AndyMasley @MatthewJBar from an individual POV, most of us are already fairly disempowered. the government has superhuman power over us: it decides rules and it has a monopoly on violence to enforce them. if AGI were to take over and run things similarly, the main losers would be officials not citizens.

@MatthewJBar fwiw I have seen more engagement with the idea and take it pretty seriously. I see the risk as more "This could be really bad" not "This definitely will" #ai-systems-are-nicer-than-humans-in-expectation" target="_blank" rel="nofollow noopener">forethought.org/research/human…

The best introductions to the three big AI risks people into EA worry about are just the 80,000 Hours articles on each: 1) Power seeking AI: 80000hours.org/problem-profil… 2) Gradual disempowerment: 80000hours.org/problem-profil… 3) Catastrophic misuse: 80000hours.org/problem-profil… Take it or leave it, agree or disagree, but if you want to know where EA people working on AI risk are coming from, these three blog posts together explain it all.

@hadivafaii isn't Bennett's argument circular though, since the Landauer limit is downstream of the second law? i never bought his argument (elaborated a bit starting on p14 of sites.pitt.edu/~jdnorton/pape…)

The blueprint for this "grand unification" already exists: --------- 🔹1961: Landauer established the thermodynamic cost of bit erasure (ieeexplore.ieee.org/document/53924…) 🔹1982: Bennett resolved Maxwell’s Demon, proving that logical irreversibility (erasure), rather than measurement, necessitates thermodynamic work (link.springer.com/article/10.100…) 🔹Late 1990s: Jarzynski and @gavincrooks related non-equilibrium work to equilibrium free energy. Their Fluctuation Theorems were later used to derive the entropy production cost of statistical inference (journals.aps.org/prl/abstract/1…; journals.aps.org/pre/abstract/1…) 🔹2007: Kawai et al. proved average thermodynamic entropy production equals the KL divergence between forward and backward path distributions( journals.aps.org/prl/abstract/1…) 🔹2009: Sagawa & Ueda incorporated information theory into this framework, bounding the energetic cost of measurement and feedback (journals.aps.org/prl/abstract/1…) 🔹2019: Wolpert extended stochastic thermo to formal computational architectures, including Turing machines and boolean circuits (iopscience.iop.org/article/10.108…) 🔹...and many more works in thermo/non-equilibrium stat mech that are waiting to be formally connected to computer science/ML concepts. --------- Formalizing this mathematical connection is a central research interest of mine. Reach out if you have ideas (DMs open).

@tamaybes a related way to spin it is that investors didn't have enough faith in the trendlines/tech/teams, and therefore got better deals early on. in a low-capital business you can outwait skepticism; in a capital-heavy business, you cannot.

It’s striking how little of the value of AI labs accrues to founders. Anthropic’s cofounders reportedly hold only ~1.8% each. Sam Altman had no equity in OpenAI for years. No founder of a frontier AI lab appears in the top 500 of the Forbes billionaires list, unless they made most of their wealth from other companies. This looks quite different from earlier tech companies. At Facebook’s IPO, Zuckerberg held ~28% of the company. At Google’s IPO, Larry and Sergey held ~16% each. I think this mostly reflects the fact that by far the most valuable factor of production in frontier AI is compute, not founders. Building a leading lab requires tens of billions in capital before meaningful revenue, and each raise dilutes founders substantially.

@krishnanrohit @giffmana Mark Lawrence did one for short stories: mark---lawrence.blogspot.com/2025/08/so-is-… One caveat is that AI writing becomes more apparent after spending time with a model. I had spent a lot more time with GPT-5 than his readers, and many of its GPT-5-specific tells were obvious to me.

@giffmana Wasn't there a paper a couple of years ago that also showed this to be the case? With respect to poetry if I remember

NYT readers prefer AI writing when tested blindly. This is a Pepsi vs CocaCola moment.

@austinc3301 @bayesianboy my steelman: the better you are at writing (and thinking), the more useless you find AI writing (and thinking). I think his day will come, but it will be later day than for the rest of us.

@bayesianboy Ted's views on AIs are surprisingly warped for a person I consider to be otherwise very intelligent

Ted Chiang responds to the article going around that asks if the left is missing out on AI: “I don’t know. Is the left missing out on ICE?”

@dylhunn @diegocabezas01 bumped context windows to 256k for thinking on Feb 20: help.openai.com/en/articles/68… and note to readers: context window is not the max input limit - that’s 128k for ChatGPT

Did you know GPT-5.4 Thinking has a 1M token context window in the API, but only 32K in ChatGPT Plus ($20/month) and 128K in Pro ($200/month)?

@ramez @Noahpinion yeah. in fairness, a very reasonable steelman is that AGI will need tons of skills and so it makes perfect sense to build training pipelines that train tons of skills. and there really is amazing generalization happening! but so far, it’s still nowhere near humans levels of G.

Yes. There's a lot of benchmarkmaxxing. I know of one case where AI models were just totally failing to succeed at a certain task (an online puzzle game). So an engineer at one of the labs built a very specific data set for that game and used it for RL. That's great. We can find problems and make AI better at them. But it's not clear it generalizes in any way, and what looks like rapid progress on benchmarks, as you say, is increasingly less about general capability increases and more about targetted work aimed at those benchmarks.

"All humans dead" sounds like a bad outcome to me

@Noahpinion @ramez although progress on these is super impressive, I think it would be radically more impressive if progress was rising as a side effect of a general training recipe and not from people trying to directly improve math, office suite, etc. hard to quantify, of course.

@ramez Well, we see rapid progress on all benchmarks, right? Like GDPval, or ARC-AGI, or First Proof, and so on?

@Miles_Brundage there’s a relationship, but they’re not equal. part of the reason for the difference is that API is frozen, so that developers get reliable operation. ChatGPT is not frozen, as we might tweak them after launch (e.g., in January we gave a speed boost to standard).

Is there any relationship between “extended” and “heavy” thinking in the ChatGPT interface and “high” and “xhigh” in the API?

@CJKRaymond you make a good point, I’m convinced

fwiw if oai people get weirdly defensive or tribal about the gullible comment I think that’s lame. have to have thicker skin then that

@robertwiblin @MatthewJBar I do think there’s a bias from: - models are necessarily simple - in a simple world there is less heterogeneity to slow things down - so most models are biased towards fast acceleration also, much progress comes from data, not algos. making data to fix your blindspots is hard.

@MatthewJBar forethought.org/research/how-q…

I find it notable how many people think AI automating AI R&D will be sufficient to trigger a capabilities explosion. I see considerably fewer people suggest that non-R&D inputs like compute and data will bottleneck the process, even though I find that hypothesis much more likely.

@jkcarlsmith @AmandaAskell How do you trade off teaching obedience (which backfires with bad orders) versus independence (which backfires with bad judgment)?

.@AmandaAskell and I are recording an audio version of Claude’s Constitution, and we’re planning to include an additional section where we answer some questions about the document. If you have questions you’re especially curious about, feel free to drop them in the replies.

@MaMoMVPY we’re 3 years closer

We are no closer to AGI today than 3 years ago.

@ramez correct take 😄

I am a skeptic on fast AI takeoff and ASI and also incredibly impressed by what LLMs can do and their rate of progress.