Aaron Scher

not the point of the thread, but source for what experts think: en.wikipedia.org/wiki/P(doom)

Glad to see AI CEOs speaking publicly about their views on AGI. I think Demis is wrong about his policy prescription: it’s far too little too late. When he says the experts disagree, he means that some think 5% this tech kills literally everybody, some at 40%, some at 90%. 🧵/9

Ending this 🧵 with some places I agree with Demis: humanity needs more time to get AGI right, & we aren’t currently giving ourselves that; the current race dynamics are very bad; we may need to (I think must) slow down; our fate is not yet sealed. Glad he’s talking publicly! 9/9

Point 4: I don't think the proposed body could actually enact, verify, and enforce a slowdown; there’s ambiguity about what’s voluntary. Again, I think we need a long-term international treaty and to actually back off, not just to slow down a little.

Point 2: I agree with the attached quote that we need more time. But I think Demis’s optimism is a vibe, not a trustworthy basis for predictions. Rob Miles says it best in this video, if an asteroid we’re headed earth’s way 200 years ago, we’d just die 🤷 youtu.be/JD_iA7imAPs?si…

That shouldn’t and can’t be the US alone. We should aim for an international agreement including the US and China, backed by verification measures and enforced like we do for other WMD proliferation. My team’s draft proposal: arxiv.org/abs/2511.10783

@dixiidev @MichaelTrazzi I’m not this person, but I like this sign. The race to ASI endangers everybody’s life and future. I don’t expect to live another 15 years—that means no having kids + watching them graduate high school. I’ve written some of my thoughts about this here aaronscher.substack.com/p/the-audacity…

@MichaelTrazzi "Stop the AI race" "I want kids" What ?

On our way to Anthropic!

@taoroalin I think that’s right if we take magnitude of impact as the denominator. It’s unclear to me if the work at frontier companies right now is net positive or not. By contrast, the vast majority of academic work is certainly positive but of smaller magnitude.

@aaronscher This feels wrong to me, the denominator should be importance or impact weighted and then the majority is at labs

Attending conferences like ICML is a good reminder that the vast majority of AI research is completely unrelated to the reckless, insane race to superintelligence being carried out by the frontier AI companies.

To enforce an AI pause while allowing AI inference, governments would need a way to distinguish training from other workloads on GPUs and other processors. We’ve just introduced a way to do this that is privacy-preserving, zero-overhead, and adversarially robust👇

Interesting choice of headline

@tszzl @hamandcheese The situation is very bad

software only intelligence explosion is possible and very likely. algorithmic progress vastly outstrips humanity’s compute buildout, and is self accelerating

happy 250th. I am forever in awe of the genius, virtue and luck it took to build something that endured this long and grew this much. I suspect we'll need just as much strength for our own time.

🇺🇸🇺🇸🇺🇸 We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness. 🇺🇸🇺🇸🇺🇸