TriProf รีทวีตแล้ว
So many people are confused about the relation between human cognitive errors and LLM hallucinations that I wrote this short explainer:
Humans say things that aren't true for many different reasons
• Sometimes they lie
• Sometimes they misremember things
• Sometimes they fail to think through what they are saying
• Sometimes they are on drugs
• Sometimes they suffer from mental disorders
etc
LLMs errors result from 𝙖 𝙙𝙞𝙛𝙛𝙚𝙧𝙚𝙣𝙩 𝙪𝙣𝙙𝙚𝙧𝙡𝙮𝙞𝙣𝙜 𝙥𝙧𝙤𝙘𝙚𝙨𝙨. They don't have (e.g.,) intentions, egos, or financial interests, so they don't lie. They don't take drugs. They don't have emotional states.
Instead, LLM "hallucinations" arise, regularly, because (a) they literally don't know the difference between truth and falsehood, (b) they don't have reliably reasoning processes to guarantee that their inferences are correct and (c) they are incapable of fact-checking their own work. Instead, everything that LLMs say -- true or false -- comes from the same process of statistically reconstructing what words are likely in some context. They NEVER fact-check what they say. Some of it is true; some is false. But even with perfect data, the stochastic reconstructive process would still produce some errors. The very process that LLMs use to generalize also creates hallucinations. (In my 2001 book I explain what a different generalization process might look like.)
§
Importantly, the goal of AGI is not to recreate humans; we don't want AGI to lie or suffer from psychiatric disorders, for example. Rather, the goal of AGI should be to build machines that can reliably reason and plan about a wide swathe of the world. The fact that humans sometimes make errors, sometimes deliberately, sometimes accidentally, in no way takes away from -- or repairs -- the limitations of the current approach.
The field of AI will eventually do better, but probably with an AI that is structured differently, in which facts are first-class citizens, rather than something you hope you might get for free with enough data.
TL;DR: Don't console yourself with making something that superficially looks like human errors, if you aspire to AGI.
English
