Jamelle Watson-Daniels

*Future Work* The framework can be extended beyond the single-turn setting to multi-turn, agents, etc. We see this as a baseline to build upon

*Reflections* as AI advances... I am seeing processes and systems previously established breaking down all around us; I want to see more people talking openly about this instead of just disengaging or pretending its business as usual

We introduce SCRuB (Social Concept Reasoning under Rubric-Based Evaluation), a framework to measure the degree to which a model reasons about social concepts with depth and critical rigor 📄 arxiv.org/abs/2605.06444 <-- arxiv paper here!

Shoutout to our incredible organizing team for yet another successful workshop @prakhar_24 , @zeyu1tang , @adoubleva , @miriamrateike , @JamelleWD , @gfarnadi , @JessicaSchrouff , @sanmikoyejo

for my pluralism homies, we added a basic pluralistic alignment eval to the report, not perfect but i think maybe the first frontier lab to do so (?)

We want your work on fairness, alignment, and/or agentic systems!! Proud to be co-organizing AFAA 2026 at ICLR @iclr_conf with: @prakhar_24 @adoubleva @miriamrateike @JamelleWD @gfarnadi @JessicaSchrouff @sanmikoyejo CFT details: afciworkshop.org #AFAA2026 #ICLR2026

website here --> prakharg24.github.io/multiplicity-t… Who’s Behind the Tutorial? We have a collaboration with a great group of researchers: Prakhar Ganesh (in person) Carol Long Afaf Taïk Hsiang Hsu Jamelle Watson-Daniels (in person) Kathleen Creel Flavio Calmon Golnoosh Farnadi

Headed to Singapore for #AAAI2026 Excited to co-present our tutorial "The Many Faces of Multiplicity in ML" Why does multiplicity matter? It helps reveal trade-offs in fairness, interpretability and reliability. Come by! Jan 20 2p Opal 107 | Singapore EXPO

Want to teach AI agents to use apps like humans? Get started with digital agents research using OpenApps, our new Python-based environment.

Release Day 🎉 Meet OpenApps — a pure-Python, open-source ecosystem for stress-testing UI agents at scale. Runs on a single CPU. Generates thousands of unique UI variations. And it reveals just how fragile today’s SOTA agents are. (Yes, even GPT-4 and Claude struggle.)

Today we're releasing Community Alignment - the largest open-source dataset of human preferences for LLMs, containing ~200k comparisons from >3000 annotators in 5 countries / languages! There was a lot of research that went into this... 🧵

I know internship hunting has been especially tough this year -- I hear you! 📢Great news: Our team at Microsoft Research (@MSFTResearch) has opened up a few additional internship slots for this summer! We're seeking talented PhD candidates with experience in 🖼️Multimodality, 🕹️Reinforcement Learning (RL), or 🤖Agents. If you're interested, I'd love to connect. Please share your CV or personal website by emailing me directly at zhang.sheng@microsoft.com or through DM. Help spread the word to those still searching for internship opportunities!

In the physical world, almost all information is transmitted through traveling waves -- why should it be any different in your neural network? Super excited to share recent work with the brilliant @mozesjacobs: "Traveling Waves Integrate Spatial Information Through Time" 1/14

Yesterday we celebrated the 5th edition of our #NeurIPS fairness workshop #AFME2024! Huge thanks to the incredible co-organizing team @adoubleva @gfarnadi @nandofioretto @JamelleWD! 🙌

📆 AFME workshop: Sat, Dec 14 in room 111-112 Excited to bring you another opportunity for insightful discussion at the AFME workshop. 💬 Join us for roundtables on Fairness/Bias Metrics and Evaluation led by the amazing @ang3linawang , @candacerossio, @tom_hartvigsen

🎊 The Algorithmic Fairness workshop #AFME2024 is one week away! #NeurIPS2024 📆 Join us Sat, Dec 14 for a day full of discussions on all things fairness and bias! Here is a preview of our amazing speakers @HodaHeidari, @krvarshney, @sanmikoyejo, @ang3linawang, @sethlazar 👇