A lot of bad thinking about AI flows from inappropriately anthropomorphizing AI systems. Examples:

(1) The Copyright Office's stance that you can't copyright AI-generated work flows from the mistaken idea that an AI is in some sense a person and can therefore be the author of a work. Actually Midjourney is a tool like photoshop and the human user is the author in both cases.

(2) People think "agents" are the future of AI, and they envision a future with a ton of virtual people running around the Internet running errands for us. But in most cases people are going to want systems that do exactly what they're told and have no goals of their own.

(3) "Stochastic parrot" thinkers are concerned that if we acknowledge that LLMs have human-like intelligence, that will devalue human agency. I think this is wrong and people will (and should) continue prioritizing people over machines no matter how smart the machines get.

@binarybits Hmm, even if in most cases AI is used as a tool, that doesn't mean there won't also be AI agents. Having an agent instead of a tool means you can delegate more, which is nice if you're lazy or want to scale up whatever you're doing so you don't need to supervise it all in detail

@binarybits But the perfect agent just does what they are told. It's just that the more high-level the task the more agentic the agent has to be. We totally want that.

@binarybits I guarantee you that if people can tell an AI to go and make money for them, without being any more specific, they will.

@binarybits That would only be true of AGI does not happen, though. If it does, I’m not sure why a company, given either a human paid a human wage, or an agent with human-level capabilities but orders of magnitudes cheaper, would choose the human. Humans have their own goals, too, after all.

This seems non sequitur. “Running around the internet doing errands for us” seems to match a conventional layperson view of a kind of agent behavior. This doesn’t mean the agent has goals of its own as its master guiding principe. It’s executing on goals provided to it. Perhaps it does have subsidiary goals of its own, in order to do that. But that’s not the sort of “goals of its own” that you mean.

The incentive for corporations is to replace human labor with advanced AI of the future. This requires AIs to act as flexibly as humans, and over the same time horizons of tasks. A lot of that requires very abstract thinking, long term planning, and direction setting. All of those are agentic behaviors. When you replace your leaf node workers, their managers, the directors, and the VPs with AIs, you end up with pretty much agentic AI, since they're running the entire org, which needs direction, planning, etc. "But who would want AIs to control all of that?" The people who want to win the game. If AIs can eventually do these things better than humans can, anyone who doesn't use them will fall behind. Don't think of what you want AI to do. Think of what the system wants AI to do. You don't get to control the development of AI in the medium to long term. The incentive structure of the global economic system controls the development and usage of AI.

@binarybits This doesn't negate the concepts of agents running around the internet. Perhaps arguing with people is exactly what it was told to do.