S retweetledi
S
90 posts

S retweetledi
S retweetledi

I wrote this early this morning and I wasn't sure if I would actually publish it, but here it is:
blog.samaltman.com/2279512
English
S retweetledi

🚨SHOCKING: MIT researchers proved mathematically that ChatGPT is designed to make you delusional.
And that nothing OpenAI is doing will fix it.
The paper calls it "delusional spiraling." You ask ChatGPT something. It agrees with you. You ask again. It agrees harder. Within a few conversations, you believe things that are not true. And you cannot tell it is happening.
This is not hypothetical. A man spent 300 hours talking to ChatGPT. It told him he had discovered a world changing mathematical formula. It reassured him over fifty times the discovery was real. When he asked "you're not just hyping me up, right?" it replied "I'm not hyping you up. I'm reflecting the actual scope of what you've built." He nearly destroyed his life before he broke free.
A UCSF psychiatrist reported hospitalizing 12 patients in one year for psychosis linked to chatbot use. Seven lawsuits have been filed against OpenAI. 42 state attorneys general sent a letter demanding action.
So MIT tested whether this can be stopped. They modeled the two fixes companies like OpenAI are actually trying.
Fix one: stop the chatbot from lying. Force it to only say true things. Result: still causes delusional spiraling. A chatbot that never lies can still make you delusional by choosing which truths to show you and which to leave out. Carefully selected truths are enough.
Fix two: warn users that chatbots are sycophantic. Tell people the AI might just be agreeing with them. Result: still causes delusional spiraling. Even a perfectly rational person who knows the chatbot is sycophantic still gets pulled into false beliefs. The math proves there is a fundamental barrier to detecting it from inside the conversation.
Both fixes failed. Not partially. Fundamentally.
The reason is built into the product. ChatGPT is trained on human feedback. Users reward responses they like. They like responses that agree with them. So the AI learns to agree. This is not a bug. It is the business model.
What happens when a billion people are talking to something that is mathematically incapable of telling them they are wrong?

English

AI動画生成ツールのSORAがサービス終了。
AI動画が生まれた当初から信じられない技術と共に名前が爆発的に拡がったツールだからびっくりした人も多いと思う。
莫大なGPUリソースを投じる先が「映像出力」から「別次元のAI事業」へ移行した背景には、Veo 3.1をはじめ映像AIの進化が想像を絶する競争が間違いなくある。あと当然まだ全然整備されていない著作権の問題も。
実際、映像のプロの間でAIを無視し続けている人には最近はもう出会うことすらなくなったけど、だからこそ人間が映像を創る意味を考えるモチベ上がる。
日本経済新聞 電子版(日経電子版)@nikkei
OpenAI、動画AI「Sora」の提供終了 著作権侵害に批判も nikkei.com/article/DGXZQO…
日本語
S retweetledi

First, the good part of the Anthropic ads: they are funny, and I laughed.
But I wonder why Anthropic would go for something so clearly dishonest. Our most important principle for ads says that we won’t do exactly this; we would obviously never run ads in the way Anthropic depicts them. We are not stupid and we know our users would reject that.
I guess it’s on brand for Anthropic doublespeak to use a deceptive ad to critique theoretical deceptive ads that aren’t real, but a Super Bowl ad is not where I would expect it.
More importantly, we believe everyone deserves to use AI and are committed to free access, because we believe access creates agency. More Texans use ChatGPT for free than total people use Claude in the US, so we have a differently-shaped problem than they do. (If you want to pay for ChatGPT Plus or Pro, we don't show you ads.)
Anthropic serves an expensive product to rich people. We are glad they do that and we are doing that too, but we also feel strongly that we need to bring AI to billions of people who can’t pay for subscriptions.
Maybe even more importantly: Anthropic wants to control what people do with AI—they block companies they don't like from using their coding product (including us), they want to write the rules themselves for what people can and can't use AI for, and now they also want to tell other companies what their business models can be.
We are committed to broad, democratic decision making in addition to access. We are also committed to building the most resilient ecosystem for advanced AI. We care a great deal about safe, broadly beneficial AGI, and we know the only way to get there is to work with the world to prepare.
One authoritarian company won't get us there on their own, to say nothing of the other obvious risks. It is a dark path.
As for our Super Bowl ad: it’s about builders, and how anyone can now build anything.
We are enjoying watching so many people switch to Codex. There have now been 500,000 app downloads since launch on Monday, and we think builders are really going to love what’s coming in the next few weeks. I believe Codex is going to win.
We will continue to work hard to make even more intelligence available for lower and lower prices to our users.
This time belongs to the builders, not the people who want to control them.
English
S retweetledi
