Siting Li

Excited to share that our paper "Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder" is accepted to #ACL2025! Preprint: arxiv.org/pdf/2411.05195 Thank @SimonShaoleiDu and @PangWeiKoh so much for your support and guidance throughout the journey!

Many people are using RL to make models smarter. We used RL to pull training data out of the models themselves. Our results show that models know a lot more about their training data than most people think. We develop Active Data Reconstruction Attack (ADRA) — a data detection method that uses RL to induce models to reconstruct data seen during training. ADRA beats existing methods by an average of >10% across pre-training, post-training, and distillation. Our paper, with @uwnlp, @Cornell, and @BerkeleyNLP @Berkeleyai, is now available. Arxiv: arxiv.org/pdf/2602.19020 Joint work with @jxmnop @shmatikov @sewon__min @HannaHajishirzi

Introducing ⚓ 𝗔𝗻𝗰𝗵𝗼𝗿𝗲𝗱 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴: a copyright mitigation strategy for any language model! With @uwnlp LMs today reproduce copyrighted text—raising concerns for creator consent and potential legal (and 💸 💸) liabilities for AI developers. 🫠 𝗔𝗻𝗰𝗵𝗼𝗿𝗲𝗱 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴 relies on two off-the-shelf LMs: 🧼A 𝘀𝗮𝗳𝗲 𝗟𝗠 trained only on permissively licensed text, ⚠️A higher-utility 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 trained on any data. The 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 drives generation, but the 𝘀𝗮𝗳𝗲 𝗟𝗠 acts as an anchor. If the 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 drifts into memorization, the 𝘀𝗮𝗳𝗲 𝗟𝗠 pulls it back ↩️. 🤝We provide a formal guarantee: outputs stays within a user-set budget of the 𝘀𝗮𝗳𝗲 𝗟𝗠. Details below! 👇 [1/⚓]

8B model can outperform AlphaEvolve on open optimization problems by scaling compute for inference or test-time RL🚀! ⭕Circle packing: AlphaEvolve (Gemini-2.0-Flash/Pro) : 2.63586276 Ours (DeepSeek-R1-0528-Qwen3-8B) : 2.63598308 🔗in🧵 [1/n]

I will be in San Diego on 12/1-12/8 and present this poster on Friday 11am-2pm. Happy to chat about multi-modal learning / unified models / other interesting stuff!

arXiv link: arxiv.org/abs/2505.15877 GitHub repo: github.com/lst627/COCO-Fa… Joint work with Xiang Gao (zcszcs522.github.io) and @SimonShaoleiDu!

🔍Image retrievers like CLIP focus on global alignment. What if we want to search by time of day, weather, or a specific gesture? 🚀 Check our paper at #NeurIPS2025! "Highlighting What Matters: Promptable Embeddings for Attribute-Focused Image Retrieval"

🔥Thrilled to introduce DR Tulu-8B, an open long-form Deep Research model that matches OpenAI DR 💪Yes, just 8B! 🚀 The secret? We present Reinforcement Learning with Evolving Rubrics (RLER) for long-form non-verifiable DR tasks! Our rubrics: - co-evolve with the policy model - are grounded on search knowledge 🧵

OpenAI's blog (openai.com/index/why-lang…) points out that today’s language models hallucinate because training and evaluation reward guessing instead of admitting uncertainty. This raises a natural question: can we reduce hallucination without hurting utility?🤔 On-policy RL with our Binary Retrieval-Augmented Reward (RAR) can improve factuality (40% reduction in hallucination) while preserving model utility (win rate and accuracy) of fully trained, capable LMs like Qwen3-8B. [1/n]

RL is bounded by finite data😣? Introducing RLVE: RL with Adaptive Verifiable Environments We scale RL with data procedurally generated from 400 envs dynamically adapting to the trained model 💡find supervision signals right at the LM capability frontier + scale them 🔗in🧵 [1/n]

The Maximal Update Parameterization (µP) allows LR transfer from small to large models, saving costly tuning. But why is independent weight decay (IWD) essential for it to work? We find µP stabilizes early training (like an LR warmup), but IWD takes over in the long term! 🧵

Forget modeling every belief and goal! What if we represented people as following simple scripts instead (i.e "cross the crosswalk")? Our new paper shows AI which models others’ minds as Python code 💻 can quickly and accurately predict human behavior! shorturl.at/siUYI🧵

WHY do you prefer something over another? Reward models treat preference as a black-box😶‍🌫️but human brains🧠decompose decisions into hidden attributes We built the first system to mirror how people really make decisions in our #COLM2025 paper🎨PrefPalette✨ Why it matters👉🏻🧵

🤔 How do we train AI models that surpass their teachers? 🚨 In #COLM2025: ✨Delta learning ✨makes LLM post-training cheap and easy – with only weak data, we beat open 8B SOTA 🤯 The secret? Learn from the *differences* in weak data pairs! 📜 arxiv.org/abs/2507.06187 🧵 below

Web data, the “fossil fuel of AI”, is being exhausted. What’s next?🤔 We propose Recycling the Web to break the data wall of pretraining via grounded synthetic data. It is more effective than standard data filtering methods, even with multi-epoch repeats! arxiv.org/abs/2506.04689

🧠 Your LLM should model how you think, not reduce you to preassigned traits 📢 Introducing LoRe: a low-rank reward modeling framework for personalized RLHF ❌ Demographic grouping/handcrafted traits ✅ Infers implicit preferences ✅ Few-shot adaptation 📄 arxiv.org/abs/2504.14439

🎉Our Spurious Rewards is available on ArXiv! We added experiments on - More prompts/steps/models/analysis... - Spurious Prompts! Surprisingly, we obtained 19.4% gains when replacing prompts with LaTex placeholder text (\lipsum) 😶‍🌫️ Check out our 2nd blog: tinyurl.com/spurious-prompt