Katja Grace 🔍

Message I sent to my family about the time-sensitive opportunity to maybe cheaply escape natural death this month: As a heads up: Some of my friends are signing up for a new procedure that can be used to chemically put the brain and body in deep freeze and potentially revive you later. It's something I'd generally recommend for older people (e.g. 70+) and terminally ill people. The tech doesn't exist today to revive people, but it seems as though enough information is preserved in the brain that medical technology will eventually advance to the point of enabling revival. (Assuming humanity doesn't destroy itself first, anyway.) I'd put this in the category of "if it weren't new and it weren't weird / outside-the-box, it would probably be standard-of-care as a last line of resort for people who medical science can't otherwise save". There are plenty of other medical procedures that are similarly risky or experimental, but that buy you far fewer years of healthy lifespan if they succeed. The biggest risks and downsides, from my perspective, are: (a) The company doing this, Nectome, is new and untested, and might turn out to be incompetent or dysfunctional in some not-yet-obvious way. (b) If it takes medical technology a long time to reach the point of being able to revive people, then Nectome might stop existing first, or some natural disaster might occur, etc. to damage or destroy the bodies. (c) Nectome only does preservation with advance notice, so you're out of luck if you pass away in a sudden accident. Some more info: - A write-up on Nectome, plus some high-quality discussion (from people I broadly respect) in the comments: [LW link] - A more general (and fun) write-up on this whole approach to end-of-life care: [@waitbutwhy link] (note that this is a ten-year-old post, and the tech was worse at the time). Per [Nectome link], Nectome's preservation services normally cost $250,000, but until April 30 they're doing a pre-sale where you can buy a $20,000 card that makes the procedure cheaper the longer you wait to use it. E.g., if you pass away in 10+ years the total cost is just the flat $20,000; if it's in 6-7 years, it's $20,000 plus an additional $90,000; etc. The card can be freely transferred at any time to anyone who needs these services, so you could potentially buy several and give them to friends and family as needed. Overall: weird stuff, but weird and neglected innovations like these are sometimes where the biggest surprises turn up. I don't think this is a super safe or ironclad bet, but I'd guess it's worth the cost if you generally care a lot about your lifespan and healthspan.

Not picking on Rohit, but: a prominent forecaster just said "I think we're probably a few months off from the world ending". Someone responded with "amusement" at the update dynamics. Nobody blinked an eye at any of this; all fully normal. Something seems incredibly wrong here.

I was also confused by this - can any accelerationists steelman these arguments as countering the typical x-risk concerns?

Ah, I may have misunderstood - I do agree that if they are mistaken about the incentives (e.g. believe there is no risk to them, or that the reward is a high chance of world domination) they may want to race regardless (and then the question is whether other parties can organize to stop that). But I think there's a good chance of China acting according to their incentives.

Yeah, checked all given solutions in the substack and the time article. I can’t see any 100% no cheating possible solutions in there. So I’m curious why you would disagree? As long as cheating is possible, and world domination is the award (or people think it is, plenty of power hungry “leaders” that do not believe AI will cause societal collapse), people will cheat.

Hell yeah!

I hate to say it but an international agreement between the US and China to ban superintelligence is inevitable. Leaders in these countries are just going to follow their incentives, and none of them are willing to give up control to an artificial superintelligence.

Something I think is important/urgent, that anyone in the US can do: watch The AI Doc and tell others about it. People relate to documentaries differently than other films. Word-of-mouth is very important. Bring more people into the conversation about AI

@interminded Not intended as US-specific, but also I disagree with your analysis. You can some of my thinking here, at 'the arms race model and its alternatives': worldspiritsockpuppet.substack.com/p/lets-think-a… or here: time.com/6283609/artifi…

@KatjaGrace Hell no! ;) Joking aside: I fully agree on pausing AI. But pushing US AI to be paused or stopped while China does not, will effectively result in Chinese world domination and the end of freedom in the rest of the world. And that’s the best outcome.. Stop all or stop none.

It's sad that the only way to prevent nuclear proliferation was to subjugate the world to the iron fist of the International Atomic Energy Agency. I just wish there was some viable alternative to living under the surveillance of this global totalitarian regime

“We cannot sit back and allow a handful of billionaire Big Tech oligarchs to make decisions that will reshape our economy, our democracy and the future of humanity." Story: rollingstone.com/politics/polit…

Dean has it backwards here. The more worried we are that current govts will misuse powerful AI for authoritarian surveillance and control, the more we should want to prevent or defer its development. It'll be harder to stop govts from using it after it's built and deployed.

Powerful speech from Cal professor @wfithian outside of the xAI office: "We're not asking them for courage." "We're not asking them to stop."  "Instead, we are here to ask them for the bare minimum."

The race to superintelligence relies on massive physical infrastructure, so it can't go on without the public's consent. We can insist firms wait until they can do it safely. One day we'll look back in wonder at how many forgot about regular people's agency. PauseAI reminds us.

Yesterday, I attended the largest AI protest in US history. I made a sign that said the following (compressed slightly): "Three of the four most cited AI experts warn of human extinction if the AI race continues. The race currently relies on brittle supply chains and city-sized data centers. Therefore the US and China can, and must, put in place an international ban on superhuman AI." All of our families are in serious, near-term danger if the race continues. Contact your elected representatives and let them know that we need urgent action on this, now.

@mattyglesias I'm most surprised by only 23% saying spanking children is morally wrong! I had thought it was popular fifty years ago then opinion had shifted much more strongly against.

[1] for details: time.com/6283609/artifi…

I wouldn't give those 'racing' for AI so much credit as to agree that it's a race [1] but whatever it is, it seems like it should be a huge priority to negotiate out of it! Protest in SF today, asking AI company leaders to agree to pause IF others will: stoptherace.ai

In the SF bay area? Join us Saturday! Make your voice heard and hear from leaders in the AI risk and safety space.

@NoamNelke @DavidSKrueger Are billions of people in a position to build 'a god'?

@KatjaGrace @DavidSKrueger If you need only a handful of people (out of billions), it doesn’t matter. You can never get to 100% and even if you could, you couldn’t keep it there forever.

Saying "AI is not inevitable" seems to have started a bit more of a conversation that I would've thought. It seems a lot of people are really insistent that it's inevitable. It's weird to witness people forcefully rejecting their agency. 🤷