Eric Neyman

@dsavitt @YafahEdelman Do you think 5/6 of the remaining millennium problems are decidable? I'm pretty close to 50/50 on that (but don't know much)!

@ericneyman @YafahEdelman That seems a touch pessimistic to me, but a better guess than all but one by 2032. Unless Hodge gets knocked off by being false, say.

I think my median for all but one of the millennium prize problems solved is 2032.

I talked with @AlexBores about why the AI industry is spending millions of dollars to defeat him: youtu.be/pydNtjE03sc?si…

I mean, an encyclical is in fact an essay...

Could an AI company lose control of its own agents? To find out, Anthropic, Google, Meta, and OpenAI let us (1) test their best internal models with CoT access, (2) review non-public info about capabilities, alignment, and control. The result: our first Frontier Risk Report.

Some personal news: I've started a new AI safety standards org, and our first two standards are out today. We're called Guidelight, co-founded with fellow ex-OpenAI safety researcher, Page Hedley. (1/n)

Really impressed with this work: xAI is the least safe of the AI companies that are near the frontier, and it's good to bring their low safety standards to the public eye.

They need to win 13 seats in 2026 and 2028. In 2026, they/independents could win any of: ME, NC, FL, OH, IA, TX, NE, MT, AK. In 2028, they could win any of: NC, WI, IA, AK. Plus maybe some independents will run? Seems extremely tricky, unfortunately.

@PoliticalKiwi Fillibuster proof majority in 2029????

Crazy poll result. If Trump's presidency does end up as a political disaster, we'll probably look back at this poll as one of the early pieces of data that confirmed it. Democrats are +10 in the generic ballot (rounded) among RV, +14 among those likely/v certain to vote in Nov

@niplav_site @MaskedTorah @GuiveAssadi According to Claude, it's enough to colonize the galaxy but not the universe. I do expect Dyson spheres, to be clear -- I just think we won't have them for thousands of years.

@ericneyman @MaskedTorah @GuiveAssadi Hm, interesting. Does fusing material on earth give enough energy to colonize the universe? Though I guess that's definitely also a ZIRP

I don’t think there will be a Dyson sphere in the 2030s or 2040s

@GuiveAssadi Hmm there's a difference between modes of thinking about this, where you're more reference-class-y, whereas Will's argument (if fleshed out) would look something more like forward-chaining / writing a story.

(1) The prior from the roughly 40T× increase in energy budget across the entire history of life means we need a lot of evidence to believe in 20T× growth in the next 20 years. (2) Regarding drosophila, growing in some localized setting is different from growing the entire energy budget of the world. (3) The world economy, in its entire history, hasn’t expanded by anywhere near 13 OOMs, so it seems unreliable to extrapolate that far. (4) I accept the argument that making labor fully substitutable with capital would dramatically increase growth, but that doesn’t establish that we’ll get 20T× growth.

@willmacaskill @GuiveAssadi @pvllss +1 to this

Genuine Q: Do you have an argument beyond "seems kind of unlikely"? I.e. which responds to the idea that you get super-exponential growth from a fully-automated economy, with a max speed of ~1 week doubling times seeming physically very feasible, given Drosophila? forethought.org/research/the-i…

@MaskedTorah @GuiveAssadi And so, Dyson spheres are a "low interest rate phenomenon": you only build them once your time discount rate on energy is like .01%/year or something. And that won't happen until well into the singularity.

@MaskedTorah @GuiveAssadi Something like: it would take tens of thousands of years to get as much power from the Sun via a solar panel than you could get via fusing the materials that comprise the solar panel into iron.

(When I first saw this tweet, I thought it was asking whether it's ok for EAs to give some of their donation budget to global health charities, given that longtermist considerations suggest that such donations aren't the most effective.)

Obviously not: 100% of your giving must be to x-risk reduction!

After much contention, my college friend group Discord server split its #ai channel into two channels: one called ai-normal for discussing AI as a normal technology, and one called ai-transformative for discussing AI as a transformative technology

When Claude Mythos found zero-day vulnerabilities in every major operating system and browser, the US government was caught flat-footed. The White House stood up an emergency interagency task force. Treasury pulled bank CEOs into an impromptu meeting. The Cybersecurity and Infrastructure Security Agency (CISA) – the agency charged with protecting US critical infrastructure – and as of late April still reportedly lacked access to Mythos. This kind of surprise is preventable. The Trump admin has already tasked the Center for AI Standards and Innovation (CAISI) with building state capacity to understand and predict future national security-relevant AI developments. But CAISI has been severely underfunded. It’s currently a $15M pilot project. In a new research report, @arthurctellis and I estimate CAISI needs ~$84M to fully deliver on its mandate. In other words, for the cost of a single F-35A fighter jet, the US government could have real situational awareness on frontier AI and not be surprised by future Mythos moments. This situational awareness can be used to inform policy and asks to the AI labs, including governance surrounding model release, safeguards, know-your-customer regimes, security protocols, and product specifications. But without a detailed understanding of these models’ capabilities — what they’re good at, how effectively they discriminate between offensive and defensive activities, whether they’re securely implemented — we’re flying blind. To estimate what it’d cost to give the government these capabilities, we translated every CAISI tasking from the AI Action Plan into FTEs and dollars, calibrated against peer evaluation orgs like METR and Anthropic's interpretability team. Two scenarios: - Limited CAISI ($26M, 56 FTE) — partial coverage of its most important taskings - Equipped CAISI ($84M, 184 FTE) — full mandate The administration's FY2027 PBR already proposed $27M for CAISI, a meaningful increase, but this was before Mythos revealed the urgency of the full mandate. To close the remaining gap: - Congress can increase FY2027 appropriations + pass the EPIC Act (creates a NIST Foundation) - The Executive can reallocate NIST STRS, tap Commerce's NRE Fund, request $84M in FY2028 PBR The price tag is small relative to comparable investments. $84M is: → A medium DARPA project → ~1 hour of the Department of War's operating budget → Less than half of NIST's Information Technology Laboratory budget And it's still less than what peer governments spend on CAISI’s peer institutions, pound-for-pound. As a fraction of their overall government budgets: UK AISI: 57 ppm Japan AISI: 32 ppm Canadian AISI: 8 ppm Current CAISI is: 1 ppm For the cost of one F-35, the administration can fully fund its own AI readiness mandate and equip the US government to anticipate the next big AI breakthrough. Full report: ifp.org/funding-for-ca…