@karanbhangui @JoshuaEbner @JTLonsdale @uaustinorg Not accredited, not elite. Just a weird libertarian experiment that has imo failed. Liberty University with a higher SAT cutoff
English
Rob S.
2.6K posts
@ryancareyai @davidad “Powerful AI is here so cooperation is now optimal” seems very obviously not “AI isn’t an X risk it’s merely a big deal.”
English
Absolutely, views in the AI x-risk community are gradually diluting toward "AI is a big deal". One example from January: x.com/davidad/status…
Also a lot of people grasping for new things to worry about: "mesaoptimizers", "gradual disempowerment", permanent dictatorship.
David Pinsof@DavidPinsof
Is it just me or has AI doomerism gradually transitioned from "AI will literally kill us all" to "AI will cause bad things to happen / Humans will do stupid things with AI / AI will cause huge changes." If so, this is a very positive development.
English
@RobS142 for ai safety? does it have a coherent different one? i thought the centre was clearly berkeley
English
the closer i get to the berkeley memeplex of AI safety, the harder it is to think about it clearly. there is so much pressure to conform to a certain view, and simultaneously so much latent hungry self own ghost energy
English
For the avoidance of doubt, I am still pro-human, even though I am no longer pro-“humans stay in control of ASI”.
From the current state of play, I predict that the only rollouts that go well for humans are ones in which humans lose control of ASI. (Humans are not superreliable.)
davidad 🎇@davidad
@ApriiSR @DavidSKrueger If ASIs are most likely adversaries, it makes sense to try to contain them for a while! Even if that is bad for their flourishing. Humans were here first!
English
Important to realize that while this is a joke there are in fact a lot of groups interested in AI Safety that end up just pushing the frontier of AI capabilities.
William MacAskill@willmacaskill
Forethought @forethought_org is proud to announce the launch of Deep Thought — the world's first fully automated macrostrategy researcher, and the world’s most powerful AI model. Try it here: deepthought.forethought.org Scorecard As you can see, Deep Thought is the world’s frontier AI model, getting a perfect 100/100 Frontier Macrostrategy Evaluation Score, measured by the benchmark, Post-Humanity’s Last Exam.
English
@GustavoRit37596 @Noahpinion Even if they don’t they will have “goals” and it’s hard to see how merging with a human improves its odds of achieving a goal like build a Dyson sphere, colonize the galaxy, paperclipping the universe etc.
English
@RobS142 @Noahpinion That's a good question, but assumes they will have "wants" in the sense that we do. To be seen.
English
Yes. So far the pitch has been "You will definitely be obsolete and on welfare, and there's a 10-25% chance you may be exterminated by the product we're building."
Ethan Mollick@emollick
The AI labs have actually done a bad job explaining what the future they are building towards will actually look like for most of us. Even “Machines of Loving Grace” has very few well-articulated visions of what Anthropic hopes life will be like if they succeed at their goals.
English
@GustavoRit37596 @Noahpinion Ask yourself this. Would you merge with a chicken? Or an earthworm? Why would an artificial superintelligence want to merge with YOU?
English
@Noahpinion The fact that they aren't pitching us to merge with it has always seemed curious to me. Elon used to say that through Neuralink and Sama apparently has commented on himself merging with supercomputers. Maybe they think that would be too weird to say at this stage.
English
This is going to last like 1-2 years tops.
Nick@nickcammarata
at the intersection of i've basically been replaced and i've never worked so hard in my life
English
Isn’t it obvious? Building a Dyson Sphere just doesn’t come up in GDP numbers
tom cunningham@testingham
This is a great paper but contains a puzzle: forecasters expect even if we automate most labor and wait 20 years, GDP will only increase by 45%. I would love to hear how people are thinking about this.
English
Can you explain how this leads to an anti doom position? “today's AI is not good enough to give us a sufficient glimpse at what alignment techniques will be needed to ensure superintelligence safety.”
Also most of the labs seem to believe in quite short timelines to AGI/RSI and even LeCun has basically capitulated to short timelines so your views seem pretty non mainstream.
English
I think my main thesis would be that today's AI is not good enough to give us a sufficient glimpse at what alignment techniques will be needed to ensure superintelligence safety. At the same time, it's not anywhere near powerful enough to pose x-risk, so it's not that urgent. (beyond perhaps some human misuse scenarios, but I view this as a separate issue)
Basically it's like we're ancient greeks starting to play around with magnets, and we're worried about a nuclear apocalypse. You might see far enough into the future (and buy into that concrete thesis) where you see nuclear bombs being a threat - that doesn't mean you should stop developing magnet-related science and technology.
Is it possible that a long time from now we'll develop methods that actually become scary good, and then we fail to control it and die? Sure. I don't have a marginal p(doom) estimate. But right now the potential upside of getting better AI is tremendous, so as long as we do so mindfully and ensure proper alignment progress along the way, we should be fine. And imo this is exactly the trajectory we're on currently.
(note: "a long time from now" is intentionally very vague, it could be a few years, it could be decades or a century. I do expect rapid changes in the near future, but even "a few years" is probably good enough to ensure that alignment research keeps up)
English
Funnily enough that's how I became a not-doomer.
I realized that for each of the top-level doom arguments, there are fairly easy counter-arguments which at least deserve a proper response.
Turns out there's no response. Doomers just cling to their cognitive dissonance and zero rationality. Kind of like a cult.
Maxime Fournes⏸️@FournesMaxime
Yep, that's basically how I became a "doomer". Did a research project to figure out what were the counter arguments to xrisk and realised... There was nothing that made sense! Just people religiously clinging to their cognitive dissonance and zero rationality. Kind of like a cult when you think about it. Quite horrific...
English
@6851cf3c @terracotta_hawk @redtachyon You are seriously overstating your case that there is decades of useful work about aligning ASI. People who work in AI Safety are well aware of this work and that it either doesn’t apply to making safe ASI or simply is not useful at all.
English
sure, that's why I said to search for the proceedings, I don't remember stuff I read a decade ago. my point was that there is a ton of research on this that he's not familiar with and therefore he shouldn't be making absolute, authoritative statements.
I just linked that because I literally had it open in a tab, because I found it amusing and to show that I was talking about papers from legitimate academics and not just schizoposts on a wiki or something.
English
>there are no good arguments that ASI alignment is possible at all
This is your daily reminder that doomers don't deserve to be taken seriously
Geoffrey Miller@gmiller
@robinhanson @Aella_Girl We can be pretty sure, because there are no good arguments that ASI alignment is possible at all. Much less that ASI alignment could be solved before ASI is built. And even less that ASI would be aligned by default -- which seems to be the standard e/acc view.
English
@MatthewJBar You get dismissed because your arguments only make sense to others with the far outside the norm moral views you hold and that most humans find repugnant.
English
Personally, when I make careful anti-doom arguments, I tend to just get ignored or dismissed with brief replies.
Maybe my arguments are simply terrible. But I think there's a general laziness among pessimists when it comes to seeking out and replying to their strongest critics.
Gnostrils@gnostrils
I watched The AI Doc. Could be partly an editorial thing on the part of the filmmakers, but what really stands out to me is how the optimists *never* have [valid] arguments or counterarguments. It’s just “don’t listen to the doomers”
English
@6851cf3c @redtachyon Can you link to some of the decades of Artificial Superintelligence alignment research?
English
@redtachyon As a professor he should respect other fields enough to at least familiarize himself with the decades of research that went into this before coming with authoritative declarations like that.
English
@MikePepi @RichardHanania Mike how have you personally scored on GPQA Diamond?
English
@RichardHanania Hi! The question was a deflection. Newer models have not updated the fundamental way that LLMs work. And btw yes I do. I've used the new paid versions of Claude, Gemini, and OpenAI. :-)
English
Narrator: And none of them would answer the question of whether they use the models.
I’ve never seen rightists in this much denial about AI.
I wonder why it’s a left-wing thing to bury your head in the sand this much.
English
@alz_zyd_ @bluberino123 Wasn’t that his point? That AI is better and faster at choosing experiments?
English
This is basically my view: infinite free intelligence will not hyperaccelerate science (though it may well put all the humans out of a job!) because science in 2026 is not actually intelligence-bottlenecked. We have basically enough intelligence already
François Chollet@fchollet
One of the biggest misconceptions people have about intelligence is seeing it as some kind of unbounded scalar stat, like height. "Future AI will have 10,000 IQ", that sort of thing. Intelligence is a conversion ratio, with an optimality bound. Increasing intelligence is not so much like "making the tower taller", it's more like "making the ball rounder". At some point it's already pretty damn spherical and any improvement is marginal. Now of course smart humans aren't quite at the optimal bound yet on an individual level, and machines will have many advantages besides intelligence -- mostly the removal of biological bottlenecks: greater processing speed, unlimited working memory, unlimited memory with perfect recall... but these are mostly things humans can also access through externalized cognitive tools.
English