Matthew Barnett

@MattWalshBlog > Even if it could — which it definitely can’t — then what? You live to watch all of your friends and loved ones die No, obviously we'd all take treatments and live long lives together.

No, sorry. You’re going to die. Very soon actually, in the grand scheme of things. You have several decades at most. In 70 or 80 years, tops, nobody reading these words right now will be alive. Most will not even be remembered. This fact is so terrifying to some people that they live every second in denial, clinging to the insane hope that somehow “science” will come along and rescue them from mortality. Even if it could — which it definitely can’t — then what? You live to watch all of your friends and loved ones die and even their tombstones decay while you linger on, trembling in fear and grasping desperately onto a life that, no matter how long it lasts, you’ve already wasted? And then you get to see the Earth decay around you and the Sun burn out ? Wow. Sounds like a lot of fun. But no thanks. I don’t need to live for a million years. I just want the time I have, however long or short, to be meaningful.

To be clear, I'm not sure whether current AI could pass this test. However, I think this prediction is on track to be falsified, even if not yet. It's also how he operationalized a key premise in his AI doom argument with Bryan Caplan, so I think it's worth evaluating.

Why would we deprive today's teens of our era's greatest source of information and support? I'm guessing many Ant researchers greatly benefited from the internet as teenagers. Growing up in the countryside in difficult family circumstances, I would have never become passionate about programming without it.

@rmushkatblat @BronsonSchoen Do you think predictions are only ever worth discussing after they have been definitively falsified or confirmed? Do you think this prediction will end up being correct?

@MatthewJBar @BronsonSchoen But AIs still can't pass the kind of test he specified in the text, as many people pointed out in the replies to your tweet, so what exactly are we doing here?

To assess whether Eliezer Yudkowsky is calibrated on AI doom, it seems relevant that in 2016 he said he'd be "pretty shocked" if an AI could pass an unrestricted one-hour Turing test before the end of the world.

@BethMayBarnes I think it's unclear whether current systems would pass such a test. It depends a lot on how such a test is conducted. But I think it's ultimately going to be a wrong prediction, under just about any reasonable real-world operationalization of the claim.

@BethMayBarnes @BronsonSchoen @tamaybes @GuiveAssadi I think in these contexts it's important to communicate clearly and precisely. It matters a lot whether AI systems are 5% vs. 75% likely to be capable of making humanity go extinct within 3 years.

@MatthewJBar @BronsonSchoen @tamaybes @GuiveAssadi I dunno, seems quite possible that "fairly likely" is compatible with 5% :P (I feel a bit torn about how much to use explicit probabilities; obviously desirable to avoid miscommunications, but can feel unrepresentative of actual epistemic state)

AI risk people commonly do this thing where they interchange P(◊X) and P(X). They say “it’s likely that X is possible,” while readers hear “it’s likely that X.” One claim is obviously much weaker than the other.

@BethMayBarnes @BronsonSchoen @tamaybes @GuiveAssadi That's quite possible. :) Someone upthread said it was "pretty obvious" that you meant the claim was "fairly likely", which is not compatible with 5% credence. This suggests the language was not clear.

@MatthewJBar @BronsonSchoen @tamaybes @GuiveAssadi Ah, I'm not sure we actually disagree a ton on substance then - maybe just a communication issue? "quite possibly" is pretty compatible with 5% in my mind. Is this a "brits and americans use 'quite' differently" thing maybe?

@Moonicker No, not as far as I'm aware.

@MatthewJBar long story short, did they settle yet?

In 2002, Ray Kurzweil and Mitchell Kapor made a bet on whether the Turing Test would be passed by 2029. To determine the winner, they devised detailed terms, including the appointment of a committee to administer an actual exam, with a set of judges and human foils.

@BethMayBarnes @BronsonSchoen @tamaybes @GuiveAssadi I think it's logically possible but very unlikely (<5%).

@MatthewJBar @BronsonSchoen @tamaybes @GuiveAssadi It seems very reasonable to me to think capability progress this rapid is pretty unlikely (e.g. <15% or something) but I'd find it hard to be very confident it's not possible (e.g. <1%) - curious how confident you are here?

You can read the detailed terms here: #adjudication_terms" target="_blank" rel="nofollow noopener">longbets.org/1/#adjudicatio…

@BronsonSchoen This prediction was how he operationalized one of the three central premises of his argument with Bryan Caplan about AI doom. I think it's relevant to his track record, even if not conclusive.

You’re trying to imply much broader claims about AI risk are false because Eliezer believed in faster takeoff than seems likely. Given that I think you’re well aware this isn’t a crux of his reviews on AI risk in the long term, this seems like another “gotcha”. I expect that you’ll claim this is about epistemics or prediction records or something, but this seems like punditry.

@davidmanheim @Jsevillamol What confident statement did I make?

@MatthewJBar @Jsevillamol I haven't tried to catalog them, and agree that he's not an amazing forecasters - but you're making confident statements generalizing from a biased sample without even trying to look?

To be clear, I'm not sure if current AI could actually pass this test. A lot hinges on how such a test is conducted. I do think the prediction will ultimately end up being wrong though.

He said this in the comment section of this post: econlib.org/archives/2016/…

@davidmanheim @Jsevillamol I'd welcome a more comprehensive evaluation of his predictions. What other falsifiable predictions has he made that directly pertain to AI doom (as opposed to unrelated predictions about other topics)?

@MatthewJBar @Jsevillamol Of course it's relevant, and filtered to support your motivated position about his accuracy. If you wanted to do any kind of evaluation of his track record properly, you'd want to collect a large set and evaluate them, instead of picking out an example where he performed poorly.

@Jsevillamol This prediction was how he operationalized one of the three central premises of his argument with Bryan Caplan about AI doom. I think it's relevant to his track record, even if not conclusive.

@MatthewJBar He has previously owned this was a bad prediction, and has also has made some surprising claims that turned out to be correct eg >16% probability of IMO gold by 2025. I love holding people accountable as much as anyone, but let's not bash people based in a single example.

@BronsonSchoen @tamaybes @GuiveAssadi If that's what she meant, then I think her claim is false. Also, if this view has become the uncontroversial consensus among AI safety groups, that suggests they are being driven by poor epistemics.

@MatthewJBar @tamaybes @GuiveAssadi The straightforward reading of the claim seems pretty obvious, i.e. “systems at this capability level seem fairly likely in the next few years”.

@BronsonSchoen @tamaybes @GuiveAssadi The controversial claim is that AIs capable of making humanity go extinct will "quite possibly" arrive in the next few years. I don't know what that claim means, yet it sounds like it was meant to be alarming.