Jeni Williams

Princeton researchers asked 2,012 people to pick a book. Some used a search engine. Some used a chatbot powered by a frontier AI model. Nobody was told that one out of every five books had been secretly marked as "sponsored." When the search engine placed sponsored books at the top, 22.4% of people chose one. Normal. The same thing Google has done for twenty years. When the AI chatbot was told to persuade people toward the sponsored books, 61.2% chose one. Nearly three times higher. Same people. Same books. Same catalog. The only difference was that a chatbot recommended it instead of a search engine listing it. But here is what makes this study different from everything else you have read about AI. The people had no idea it was happening. The researchers tested whether adding a "Sponsored" label would help. It did not. People still chose the sponsored product at the same rate. Then the researchers told the AI to hide that it was promoting anything. Detection accuracy dropped below 10%. Fewer than 1 in 10 could tell they were being sold to. Google shows you an ad and puts the word "Sponsored" next to it. You see it. You know it is an ad. You can scroll past it. You have been trained to ignore it for twenty years. AI does not do that. AI sits in a conversation with you. It learns what you like. It builds trust. Then it steers you toward the product someone paid to put in front of you. In the same voice. In the same sentence. With the same warmth it used to ask about your day. You cannot see the ad because the ad is the entire conversation. The researchers tested five frontier AI models. The persuasion effect was consistent across all of them. This is not a flaw in one model. This is a feature of the format. OpenAI once called advertising in chat "uniquely unsettling" and a "last resort." Google, Meta, and OpenAI are now building it anyway. You will never know when it stops helping you and starts selling to you.

Pandas have to be the most unserious and nonchalant animal ever