A lot of bad thinking about AI flows from inappropriately anthropomorphizing AI systems. Examples:

(1) The Copyright Office's stance that you can't copyright AI-generated work flows from the mistaken idea that an AI is in some sense a person and can therefore be the author of a work. Actually Midjourney is a tool like photoshop and the human user is the author in both cases.

(2) People think "agents" are the future of AI, and they envision a future with a ton of virtual people running around the Internet running errands for us. But in most cases people are going to want systems that do exactly what they're told and have no goals of their own.

(3) "Stochastic parrot" thinkers are concerned that if we acknowledge that LLMs have human-like intelligence, that will devalue human agency. I think this is wrong and people will (and should) continue prioritizing people over machines no matter how smart the machines get.

(4) Relatedly X-riskers assume that human-level intelligence will inevitably lead to human-level ambition and a human-like survival instinct, which will drive them to try to kill us. I think it won't be hard to train models that have no goals beyond the narrow ones we give them.

In all cases I think the fundamental mistake is that we're extrapolating from a sample size of one: currently there is only one type of entity with human-level intelligence, human beings, and so we assume all other intelligent entities will have our characterstics.

But in fact I suspect that intelligence is multi-dimensional, and AI systems can be similar to (or much better than) than us in some dimensions while being totally different in others.

@binarybits I strongly agree with the claim that AI (can be / is already) smarter than humans in some ways, and dumber in others, but don't understand how it leads to the conclusion that AI won't later exceed humans on all of the dimensions. twitter.com/davidmanheim/s…

@binarybits You're making the same error by even calling the thing that has various dimensions "intelligence". The thing is with lots of aspects is "agency" or "personhood" or "liveness" or "value coherence" or "autopoetic entityhood" or various other properties of active material systems.

@binarybits Yes. For instance, AI will always be better than us in spotting connections and creating hybrids. But I don't think AI can ever emulate absolute true creativity, such as what we see when we observe an infant at play.

@binarybits The only risk is from human bad actors. Same as it ever was.

I think this is one area where the discussion definitely lacks detail. Most people talk of “intelligence” as a one dimensional metric where you either have more or less, and if you have more than you are better at everything than someone who has less. This is obviously not true in humans, and I see no reason that it will be true in AIs

@binarybits Totally. AIG should also be called "human-level or better in all things". It's already so much better in some, so much worse in other things. And really dangerous AI and generally useful AI, both can remain behind in some things. It probably does not need AGI for anything.