Xinyue Liu

🙌 Excited to share our new paper and my first project in my PhD journey! We show finetuning on a writing task unlocks verbatim recall of copyrighted books from authors not in the finetuning data. It’s been an incredible experience working with such an amazing group of people ✨

@StephanieKelton @stonybrooku Thank you! 🙏🙏🙏

Lead author @stonybrooku @irisiris_l

👀 Can AI produce a novel worth reading? We built a platform to find out. 📚 Introducing AutoFiction: a web platform that hosts AI-generated novels by Claude Code & Codex, rated and reviewed by real readers. We have 33 books so far, spanning dark fantasy, murder mysteries, Harry Potter fanfics, and more. All free to read. (1/n)

Is the Internet quietly being rewritten to serve AI agents? How do we even measure this? New paper: We find that post-ChatGPT, listings on Etsy have been systematically reshaped to influence how agents behave—without making humans worse off. We call these “mecha-nudges”.🧵

@mar_kar_ Thank you! This means a lot to me 🙏

@irisiris_l Congrats Xinyue, such a strong start!

🙌 Excited to share our new paper and my first project in my PhD journey! We show finetuning on a writing task unlocks verbatim recall of copyrighted books from authors not in the finetuning data. It’s been an incredible experience working with such an amazing group of people ✨

@nickmvincent Thank you! 🙏

@irisiris_l Really important work!

@TuhinChakr @mjbommar Thank you! 🙏🫶

@mjbommar All credit to my amazing PhD @irisiris_l

after years of being gaslit by ignorant clowns or bought industry shills, it's wonderful to see the blossoming of truth - that, no shit, minimizing next token loss results in a compressed, often lossless copy of input data. who'da'thunk?

One important finding here, besides the copyright ramifications, is that emergent misalignment can occur when you fine-tune on benign-looking data as well, and there are no alarm bells for it. You cannot always predict how your fine-tuning data can have transitive effects and what harms it can cause to other domains. This was a super fun collaboration. Check out our interactive demo: cauchy221.github.io/Alignment-Whac…

Fine-tuning *commercial models* (GPT5, Gemini, ...) on one author's data unlocks regurgitation of other authors copyrighted material!! in our new preprint, alignment whack-a-mole🦫 we show emergent misalignment for copyright and memorization! Anyone who has talked to me in the past few weeks has heard my schpeel on how memorization in LLMs is transitive with respect to some latent variable, that models learn shared representations during pretraining, and finetuning on one side of the latent unlocks the other. The latent could be anything such as authorship, style, 'badness' (emergent misalignment), or it could be "copyrighted literary text.", any co-occuring content in pre-training. Amazing work led by @irisiris_l, @TuhinChakr and with Jane Ginsburg!

@TuhinChakr 🚀🚀🚀

@irisiris_l ❤️❤️❤️❤️ LFG !!!

A very nice rundown of highlights from AI safety research in 2025 by @FabienDRoger lesswrong.com/posts/nAsMfmxD…

This is really cool. It got me thinking more deeply about personalized RL: what’s the real point of personalizing a model in a world where base models can become obsolete so quickly? The reality in AI is that new models ship every few weeks, each better than the last. And the pace is only accelerating, as we see on the Hugging Face Hub. We are not far away from better base models dropping daily. There’s a research gap in RL here that almost no one is working on. Most LLM personalization research assumes a fixed base model, but very few ask what happens to that personalization when you swap the base model. Think about going from Llama 3 to Llama 4. All the tuned preferences, reward signals, and LoRAs are suddenly tied to yesterday’s model. As a user or a team, you don’t want to reteach every new model your preferences. But you also don’t want to be stuck on an older one just because it knows you. We could call this "RL model transferability": how can an RL trace, a reward signal, or a preference representation trained on model N be distilled, stored, and automatically reapplied to model N+1 without too much user involvement? We solved that in SFT where a training dataset can be stored and reused to train a future model. We also tackled a version of that in RLHF phases somehow but it remain unclear more generally when using RL deployed in the real world. There are some related threads (RLTR for transferable reasoning traces, P-RLHF and PREMIUM for model-agnostic user representations, HCP for portable preference protocols) but the full loop seems under-studied to me. Some of these questions are about off-policy but other are about capabilities versus personalization: which of the old customizations/fixes does the new model already handle out of the box, and which ones are actually user/team-specific to ever be solved by default? That you would store in a skill for now but that RL allow to extend beyond the written guidance level. I have surely missed some work so please post any good work you’ve seen on this topic in the comments.

We invited Claude users to share how they use AI, what they dream it could make possible, and what they fear it might do. Nearly 81,000 people responded in one week—the largest qualitative study of its kind. Read more: anthropic.com/features/81k-i…

和@sainingxie 一起挑战7小时播客！他刚和Yann LeCun踏上“世界模型”的创业旅程（AMI Labs）。这是他第一次Podcast、第一次访谈。 2026年2月雪后的一天，我们在纽约布鲁克林，从下午2点，开启了一场始料未及的马拉松式访谈，直到凌晨时分散去。 这篇访谈的中文标题叫做《逃出硅谷》，但他又不厌其烦地枚举了影响他学术生涯的每一个人，并反反复复口头描摹这些人的人物特征（侯晓迪、何恺明、杨立昆、李飞飞…）正是这些，让这篇“逃出硅谷”的对话充斥着人性的温度。 By the way, 下面是访谈的YouTube版本，我们提供了中英字幕。 And yes, 我们是在用播客给这个世界建模😎 A 7-hour podcast with Saining Xie. He has just begun a new journey on world models with Yann LeCun at AMI Labs. This was his first podcast appearance and his first long-form interview. A day after the snowfall in February 2026, in Brooklyn, New York, we started recording at 2 p.m. What followed became an unexpected marathon conversation that lasted until the early hours of the morning. The Chinese title of the interview is “Escaping Silicon Valley.” Yet throughout the conversation, he patiently listed the people who shaped his academic life, repeatedly sketching their personalities in vivid detail: Hou Xiaodi, Kaiming He, Yann LeCun, Fei-Fei Li, and others. These portraits are what give this “escape from Silicon Valley” conversation its human warmth. By the way, the YouTube version of the interview is below, with Chinese and English subtitles. And yes, we are using podcasts to model the world 😎 A 7-hour marathon interview with Saining Xie: World Models, AMI Labs, Ya... youtu.be/rIwgZWzUKm8?si… 来自 @YouTube

Super cool phenomena, in my head it relates to semantic memorization and leakage, and even cross-modal leakage that i like to term “transitive” memorization. arxiv.org/abs/2408.06518 arxiv.org/abs/2507.17937

Dear followers, I’m happy to share this new academic paper on how even capable AI can lead to deterioration of collective knowledge in society

Tomorrow @dashiel_carrera @bruceholsinger @mariskreizman1 will be at the @Center4Fiction speaking about Generative AI and what it means for the literary world :) If you are in NYC and enjoying the good weather please stop by

🚨 Stanford just analyzed the privacy policies of the six biggest AI companies in America. Amazon. Anthropic. Google. Meta. Microsoft. OpenAI. All six use your conversations to train their models. By default. Without meaningfully asking. Here's what the paper actually found. The researchers at Stanford HAI examined 28 privacy documents across these six companies not just the main privacy policy, but every linked subpolicy, FAQ, and guidance page accessible from the chat interfaces. They evaluated all of them against the California Consumer Privacy Act, the most comprehensive privacy law in the United States. The results are worse than you think. Every single company collects your chat data and feeds it back into model training by default. Some retain your conversations indefinitely. There is no expiration. No auto-delete. Your data just sits there, forever, feeding future versions of the model. Some of these companies let human employees read your chat transcripts as part of the training process. Not anonymized summaries. Your actual conversations. But here's where it gets genuinely dangerous. For companies like Google, Meta, Microsoft, and Amazon companies that also run search engines, social media platforms, e-commerce sites, and cloud services your AI conversations don't stay inside the chatbot. They get merged with everything else those companies already know about you. Your search history. Your purchase data. Your social media activity. Your uploaded files. The researchers describe a realistic scenario that should make you pause: You ask an AI chatbot for heart-healthy dinner recipes. The model infers you may have a cardiovascular condition. That classification flows through the company's broader ecosystem. You start seeing ads for medications. The information reaches insurance databases. The effects compound over time. You shared a dinner question. The system built a health profile. It gets worse when you look at children's data. Four of the six companies appear to include children's chat data in their model training. Google announced it would train on teenager data with opt-in consent. Anthropic says it doesn't collect children's data but doesn't verify ages. Microsoft says it collects data from users under 18 but claims not to use it for training. Children cannot legally consent to this. Most parents don't know it's happening. The opt-out mechanisms are a maze. Some companies offer opt-outs. Some don't. The ones that do bury the option deep inside settings pages that most users will never find. The privacy policies themselves are written in dense legal language that researchers people whose job is reading these documents found difficult to interpret. And here's the structural problem nobody is addressing. There is no comprehensive federal privacy law in the United States governing how AI companies handle chat data. The patchwork of state laws leaves massive gaps. The researchers specifically call for three things: mandatory federal regulation, affirmative opt-in (not opt-out) for model training, and automatic filtering of personal information from chat inputs before they ever reach a training pipeline. None of those exist today. The uncomfortable truth is this: every time you type something into ChatGPT, Gemini, Claude, Meta AI, Copilot, or Alexa, you are contributing to a training dataset. Your medical questions. Your relationship problems. Your financial details. Your uploaded documents. You are not the customer. You are the curriculum. And the companies doing this have made it as hard as possible for you to stop.

@niloofar_mire @dylan522p 🫂

@dylan522p U think i got any sleep last night?

I was literally just about to go to sleep... now I'ma be up for the next few hours