6Glab

P(doom) roundup: what probability do people put on AI killing everyone? - Vitalik Buterin (Ethereum): 10% - Zvi Mowshowitz: 60% - Elon Musk: 20-30% - Scott Alexander: 20-25% - Dario Amodei (CEO, Anthropic): 10-25% - Jan Leike (Head of Alignment, OpenAI): 10-90% - Geoffrey Hinton (Godfather of AI): 10% - Paul Christiano (Former Head of Alignment at OpenAI, inventor of RLHF): 50% - Lina Khan (FTC Chair): 15% - Average AI engineer (Oct 2023): ~40% - Average ML researcher (in spring 2022, before things got crazy): 10% - Dan Hendrycks: recently updated from 20% to 80% - Average AI alignment researcher: 30% - Extinction tournament (median for AI experts): 20% chance of catastrophe, 6% chance of extinction - Extinction tournament (median for non-AI experts): 9% chance of catastrophe, 1% chance of extinction - BACA Research: 50% - Scott Aaronson: 2% - Conjecture AI researchers: 80% - Eli Lifland: 35% - Eliezer Yudkowsky: >95% - Nate Soares: >95% (I think?) - Holden Karnofsky: 50% - Average American: 26% Note: these are just a few I recall from memory and could easily find sources for. What’s your p(doom)? My take: I agree with @TheZvi - if your p(doom) is anywhere between 10-90%, it shouldn’t really change what you do, that’s plenty high to justify urgent action. * Disclaimers: p(doom) usually means as “extinction or similarly bad outcome” but everyone defines it differently, some of these are old and may have changed, many people added various caveats and conditionals, etc. Just squint and notice the general pattern: a few tech companies are playing Russian Roulette with humanity.