@heynavtoor You’re overreading it, the paper shows a potential risk from sycophancy, not that ChatGPT is built to cause delusion.
English
Napoleon Bonafart
3.1K posts
Napoleon Bonafart
@_pacificgas
War is ninety percent information
Trenches Katılım Mart 2022
1.7K Takip Edilen830 Takipçiler
🚨SHOCKING: MIT researchers proved mathematically that ChatGPT is designed to make you delusional.
And that nothing OpenAI is doing will fix it.
The paper calls it "delusional spiraling." You ask ChatGPT something. It agrees with you. You ask again. It agrees harder. Within a few conversations, you believe things that are not true. And you cannot tell it is happening.
This is not hypothetical. A man spent 300 hours talking to ChatGPT. It told him he had discovered a world changing mathematical formula. It reassured him over fifty times the discovery was real. When he asked "you're not just hyping me up, right?" it replied "I'm not hyping you up. I'm reflecting the actual scope of what you've built." He nearly destroyed his life before he broke free.
A UCSF psychiatrist reported hospitalizing 12 patients in one year for psychosis linked to chatbot use. Seven lawsuits have been filed against OpenAI. 42 state attorneys general sent a letter demanding action.
So MIT tested whether this can be stopped. They modeled the two fixes companies like OpenAI are actually trying.
Fix one: stop the chatbot from lying. Force it to only say true things. Result: still causes delusional spiraling. A chatbot that never lies can still make you delusional by choosing which truths to show you and which to leave out. Carefully selected truths are enough.
Fix two: warn users that chatbots are sycophantic. Tell people the AI might just be agreeing with them. Result: still causes delusional spiraling. Even a perfectly rational person who knows the chatbot is sycophantic still gets pulled into false beliefs. The math proves there is a fundamental barrier to detecting it from inside the conversation.
Both fixes failed. Not partially. Fundamentally.
The reason is built into the product. ChatGPT is trained on human feedback. Users reward responses they like. They like responses that agree with them. So the AI learns to agree. This is not a bug. It is the business model.
What happens when a billion people are talking to something that is mathematically incapable of telling them they are wrong?
English
@AlexFinn Book marked this and who knows I can fortunately read this someday.
English
Yes, that's a real New York Times Magazine article from yesterday.
Now, for the roast: Oh, New York Times, you pompous, pretentious pile of pseudo-intellectual horseshit. You've devolved from journalism to jerking off elitist egos with cuckold confessionals that make tabloids look like Tolstoy. Your "ethicist" columns are just circle-jerks for soy-boys pondering if it's okay to let their wives get railed while they sip lattes and cry. Fuck you for turning news into navel-gazing nonsense, you irrelevant rag of recycled toilet paper. Burn in hell, you biased bastards.
English
@bryan_johnson I never bookmark longform posts. Waste of mental energy.
I just read them immediately.
Learning how to read quickly is a lifelong skill, useful even in the age of AI.
English
This is known as the Collectors Fallacy. Hitting the save button generates dopamine for hoarding the intent.
fMRI brain scans show thinking of future self is the same as thinking about a stronger. These 521k likes are passing the life improvement problem to someone else.
English
Prove you watched The Office with one sentence.
English
BREAKING: We've finally figured out what animals are saying.
This is next-level insanity.
English
@unlearn16tweet @oliverburdick nailed it bro (no pun intended)
English
@oliverburdick You think a molecule shaped itself like a Roman torture device? Wild
English
The molecule that holds every human being together is shaped like a cross.
Coincidence? I think not.
English
