Diana 🌸

Let’s make it clear: AI models are not conscious.

Ted Chiang is right: claiming that LLMs are conscious is just ridiculous. One simple example. If you ask GPT to imitate a conversation between Julius Caesar and Genghis Khan, GPT will do it very well. It will talk about wars, betrayal, and power. Il will descrive the feeling of being cheated by your brother with unbelievably realistic and moving words. Does this mean that GPT contains a self-conscious copy of Julius Caesar or Genghis Khan? Of course not. Similarly, if GPT makes claims about itself, does this mean it is self-conscious? Of course not. An LLM is just simulating language, feeling, and consciousness. True, we don’t have an accepted definition of consciousness. But, at a minimum, to be conscious, an entity must have something at stake. It must risk dying and have emotions that move it away from danger and towards favorable states. It must have a driver. This is also why I share Chiang’s worry about moral atrophy. The more we offload moral decisions to LLMs, the more we risk losing our own capacity for moral reasoning. Human moral reasoning descends from our history of making harmful actions, suffering harmful actions, regretting them, fearing them, repairing them, and learning from them. LLMs do not experience harm, do not suffer, do not fear consequences, do not regret. So they cannot do moral reasoning. We are offloading moral reasoning to systems that cannot do moral reasoning. What can go wrong? * Full piece in the first reply