loic

There's a recurring pattern that I keep seeing in work coming from the apocalyptic branch of the AI safety community, and it goes something like this: 1) Pick some intellectual "talent" that might be attributed to a person (e.g. persuasiveness, discernment, charisma, etc.) 2) Model this "talent" as a scalar quantity that is primarily determined by factors endogenous to the individual, as opposed to environmental/situational factors 3) Assume that this scalar quantity can be made arbitrarily large 4) Use this model to make predictions about the future of AI You can see this here with AI 2040's insinuation that "superhuman persuasiveness" is an idea we should be taking seriously It's not obvious to me at all that "persuasiveness" is a human talent, as opposed to a sociological random process that we retroactively perceive as a human "talent" To be clear, certainly it's true that a star debater might be marginally more "persuasive" than someone who's not! But I don't think a cult leader or a popular politician is 1,000x or 1,000,000x more "persuasive" than an ordinary person Rather, they're perceived being "persuasive" because they happen to be the figureheads for a complex sociological preference cascade. Their "persuasiveness" isn't really a thing you can causally influence at the individual level, and definitely not in an unbounded way In general, I think a lot of the AI 2040-style forecasting work does a poor job of dealing with this kind of irreducible complexity inherent to the universe. They usually just like to pretend it doesn't exist Not a huge fan of this pattern

>claims positive vision for the future >ask if it's e/acc or global AI Doomer communism >Doomer is confused >I make a diagram explaining UBI vs free markets >"It's a good vision sir." >Look inside >It's Communism. >mfw 😑

The percentage of people outside of the very very small AI aware world who know that a reasonable percentage of those inside the AI aware world believe this to be true is close to zero

When technology advances, the benefits get distributed widely, regardless of who owns the main companies profiting. The discussions about AI and distribution completely miss the point on how progress works. The important thing is to not strangle the technology.

The author of the piece responds with “I am only stating a premise and suggesting what it implies.” My contention is that, no, the premise itself smuggles in the assumption of the conclusion. Let me try to give you a more concrete example. Take politics, and take a broad conception of politics. Not just what elected officials do but the art of collective persuasion, coalition-building, and related strategic effort that is embedded in every organization. The higher you rise in many professions, the likelier it is that you do politics of some form or another. This premise assumes that AI is superior to humans at every single aspect of politics. Not just coming up with political strategies or finding fulcra that might persuade specific individuals or groups, but instead executing those strategies. Actually building coalitions and persuading people itself. Not just writing speeches but delivering them (is this a coherent concept?). I submit that persuasive acts like this are not unilateral or intrinsic capabilities of the actor. The recipient of the message also has a say in whether the persuasion worked. This is why the same words, delivered by different messengers, can land differently. This is why, for example, many people on this very website who used to read my writing in a favorable way are now inclined to read it unfavorably after I announced I’d be joining OpenAI. Every single aspect of the messenger affects the message itself, or at least it affects how that message is received by others. To argue that AI would be better at every aspect of politics, then, is to assume a massive degree of institutional transformation. Many such transformations are possible, but the single likeliest one would be that the human role in deciding what organizations and societies do simply does not exist, or does not matter nearly as much. And if you believe that, it would almost be a necessity that the rule of law in general, and contractual grants of equity in particular, would not be honored, the latter being the actual thesis of the essay in question. So by saying “AI is better than humans at ALL labor,” you have in fact smuggled in what amounts to the author’s conclusion. And that is what my original tweet was saying. I stand by every aspect of it. Perhaps you believe this will happen! That’s fine. But as the writer, it’s incumbent upon you to persuade the reader of this from outside the bounds of the tautology. This essay fails in that regard, as does a great deal of other writing on this topic.