@GustavoRit37596 @Noahpinion Even if they don’t they will have “goals” and it’s hard to see how merging with a human improves its odds of achieving a goal like build a Dyson sphere, colonize the galaxy, paperclipping the universe etc.
English
Rob S.
2.6K posts
@RobS142 @Noahpinion That's a good question, but assumes they will have "wants" in the sense that we do. To be seen.
English
Yes. So far the pitch has been "You will definitely be obsolete and on welfare, and there's a 10-25% chance you may be exterminated by the product we're building."
Ethan Mollick@emollick
The AI labs have actually done a bad job explaining what the future they are building towards will actually look like for most of us. Even “Machines of Loving Grace” has very few well-articulated visions of what Anthropic hopes life will be like if they succeed at their goals.
English
@GustavoRit37596 @Noahpinion Ask yourself this. Would you merge with a chicken? Or an earthworm? Why would an artificial superintelligence want to merge with YOU?
English
@Noahpinion The fact that they aren't pitching us to merge with it has always seemed curious to me. Elon used to say that through Neuralink and Sama apparently has commented on himself merging with supercomputers. Maybe they think that would be too weird to say at this stage.
English
This is going to last like 1-2 years tops.
Nick@nickcammarata
at the intersection of i've basically been replaced and i've never worked so hard in my life
English
Isn’t it obvious? Building a Dyson Sphere just doesn’t come up in GDP numbers
tom cunningham@testingham
This is a great paper but contains a puzzle: forecasters expect even if we automate most labor and wait 20 years, GDP will only increase by 45%. I would love to hear how people are thinking about this.
English
Can you explain how this leads to an anti doom position? “today's AI is not good enough to give us a sufficient glimpse at what alignment techniques will be needed to ensure superintelligence safety.”
Also most of the labs seem to believe in quite short timelines to AGI/RSI and even LeCun has basically capitulated to short timelines so your views seem pretty non mainstream.
English
I think my main thesis would be that today's AI is not good enough to give us a sufficient glimpse at what alignment techniques will be needed to ensure superintelligence safety. At the same time, it's not anywhere near powerful enough to pose x-risk, so it's not that urgent. (beyond perhaps some human misuse scenarios, but I view this as a separate issue)
Basically it's like we're ancient greeks starting to play around with magnets, and we're worried about a nuclear apocalypse. You might see far enough into the future (and buy into that concrete thesis) where you see nuclear bombs being a threat - that doesn't mean you should stop developing magnet-related science and technology.
Is it possible that a long time from now we'll develop methods that actually become scary good, and then we fail to control it and die? Sure. I don't have a marginal p(doom) estimate. But right now the potential upside of getting better AI is tremendous, so as long as we do so mindfully and ensure proper alignment progress along the way, we should be fine. And imo this is exactly the trajectory we're on currently.
(note: "a long time from now" is intentionally very vague, it could be a few years, it could be decades or a century. I do expect rapid changes in the near future, but even "a few years" is probably good enough to ensure that alignment research keeps up)
English
Funnily enough that's how I became a not-doomer.
I realized that for each of the top-level doom arguments, there are fairly easy counter-arguments which at least deserve a proper response.
Turns out there's no response. Doomers just cling to their cognitive dissonance and zero rationality. Kind of like a cult.
Maxime Fournes⏸️@FournesMaxime
Yep, that's basically how I became a "doomer". Did a research project to figure out what were the counter arguments to xrisk and realised... There was nothing that made sense! Just people religiously clinging to their cognitive dissonance and zero rationality. Kind of like a cult when you think about it. Quite horrific...
English
@6851cf3c @terracotta_hawk @redtachyon You are seriously overstating your case that there is decades of useful work about aligning ASI. People who work in AI Safety are well aware of this work and that it either doesn’t apply to making safe ASI or simply is not useful at all.
English
sure, that's why I said to search for the proceedings, I don't remember stuff I read a decade ago. my point was that there is a ton of research on this that he's not familiar with and therefore he shouldn't be making absolute, authoritative statements.
I just linked that because I literally had it open in a tab, because I found it amusing and to show that I was talking about papers from legitimate academics and not just schizoposts on a wiki or something.
English
>there are no good arguments that ASI alignment is possible at all
This is your daily reminder that doomers don't deserve to be taken seriously
Geoffrey Miller@gmiller
@robinhanson @Aella_Girl We can be pretty sure, because there are no good arguments that ASI alignment is possible at all. Much less that ASI alignment could be solved before ASI is built. And even less that ASI would be aligned by default -- which seems to be the standard e/acc view.
English
@MatthewJBar You get dismissed because your arguments only make sense to others with the far outside the norm moral views you hold and that most humans find repugnant.
English
Personally, when I make careful anti-doom arguments, I tend to just get ignored or dismissed with brief replies.
Maybe my arguments are simply terrible. But I think there's a general laziness among pessimists when it comes to seeking out and replying to their strongest critics.
Gnostrils@gnostrils
I watched The AI Doc. Could be partly an editorial thing on the part of the filmmakers, but what really stands out to me is how the optimists *never* have [valid] arguments or counterarguments. It’s just “don’t listen to the doomers”
English
@6851cf3c @redtachyon Can you link to some of the decades of Artificial Superintelligence alignment research?
English
@redtachyon As a professor he should respect other fields enough to at least familiarize himself with the decades of research that went into this before coming with authoritative declarations like that.
English
@MikePepi @RichardHanania Mike how have you personally scored on GPQA Diamond?
English
@RichardHanania Hi! The question was a deflection. Newer models have not updated the fundamental way that LLMs work. And btw yes I do. I've used the new paid versions of Claude, Gemini, and OpenAI. :-)
English
Narrator: And none of them would answer the question of whether they use the models.
I’ve never seen rightists in this much denial about AI.
I wonder why it’s a left-wing thing to bury your head in the sand this much.
English
@alz_zyd_ @bluberino123 Wasn’t that his point? That AI is better and faster at choosing experiments?
English
This is basically my view: infinite free intelligence will not hyperaccelerate science (though it may well put all the humans out of a job!) because science in 2026 is not actually intelligence-bottlenecked. We have basically enough intelligence already
François Chollet@fchollet
One of the biggest misconceptions people have about intelligence is seeing it as some kind of unbounded scalar stat, like height. "Future AI will have 10,000 IQ", that sort of thing. Intelligence is a conversion ratio, with an optimality bound. Increasing intelligence is not so much like "making the tower taller", it's more like "making the ball rounder". At some point it's already pretty damn spherical and any improvement is marginal. Now of course smart humans aren't quite at the optimal bound yet on an individual level, and machines will have many advantages besides intelligence -- mostly the removal of biological bottlenecks: greater processing speed, unlimited working memory, unlimited memory with perfect recall... but these are mostly things humans can also access through externalized cognitive tools.
English
@LinchZhang @BenShindel The original tweet was about depressed people. Someone who is in hospice and also depressed seems quite different.
English
@BenShindel If I'm ever in extreme pain in hospice care, I aspire to have the emotional fortitude to ace PHQ-9 tests and be a remarkable ray of sunshine in every aspect except wanting to end my life.
English
@RobS142 @GarettJones @ndtippens @ChadJonesEcon the laws of physics do not permit the existence of an entity with IQ higher than Von Neumann's
English
@alz_zyd_ @GarettJones @ndtippens @ChadJonesEcon Wrong framing imo. 100 John Von Neumanns cannot solve the same types of problems a 100x IQ Super Von Neumann can.
English
@GarettJones @ndtippens @ChadJonesEcon right, so if all the low-hanging fruit is gone, another order of magnitude increase in intelligence quantity isn't going to massively accelerate science
English
@alz_zyd_ @bluberino123 You mean in general or if AI is better at choosing experiments than Humans that Humans would still decide for some reason?
English
“Trust me I’m an architect and that’s why ugly utilitarian modern buildings are actually better than beautiful traditional ones.” Architects are apparently the most pretentious and out of touch people in the world.
Bob Sacamano@PedroCo67443965
@UrbanCourtyard Call it what you want, but they are in fact imitating old buildings, which doesn't work along with functionality, materials and technology. Trust me, I'm an architect. It's popular among nostalgic and sentimental people without being full conscious about architecture.
English
@PedroCo67443965 @UrbanCourtyard Why why are traditional facades fake and bad but new ones are better and good?
English
@TSHamiltonAstro @UrbanCourtyard if you enjoy pastiche fake old facades, maybe that tells something about yourself...
English
@neil_chilson @binarybits @ATabarrok @deanwball @reason When I read critiques like this I just can’t help but think you don’t actually believe superintelligence is possible.
English
@binarybits @ATabarrok @deanwball Exactly! From my @reason review of “If Anyone Builds It, Everyone Dies”:
English
This is the correct view of existential risk from AI, and I'm glad @deanwball sees the same connection to Hayek's thinking that I do.
English
@panickssery Is this a useful definition of literacy if, under it, I could read a Spanish language newspaper out loud and be considered literate despite having zero understanding of the content?
English
Very tedious element of "literacy rate" talk is that people use the word literacy not to mean the ability to read but the ability to understand and infer
To me a good standard of literacy is whether you can read a page of an instruction manual or newspaper out loud at 60wpm
English