I struggle with what to say about the new AI 2040: Plan A website. It all seems so implausible to me that I'm not sure where to start. There's an epistemic chasm between those who think superintelligence implies near-omnipotence and those (like me) who don't.

@binarybits What do you make of the argument that the people who believe in it have been much better forecasters of what has happened so far than virtually anyone else? Shouldn't that cause you to update your priors?

@F_Vaggi Some (by no means all or even most) have a good track record of predicting how model capabilities have improved. I think "how will this impact the world" is a different question requiring different expertise to forecast well.

@binarybits AFAIK - one of the main authors is the 2nd highest forecaster. As I acknowledged, there is a huge amount of uncertainty and my "gut feeling" tells me it's implausible, but, I also have to keep repeating myself "several of these people are better than me at forecasting"