MohammadHossein Rezaei
37 posts
@mhrezaeics
Post-training Research @ScaleAILabs | Ex Research Intern @StanfordNLP | CS @UArizona
🔄RLHF → RLVR → Rubrics → OnlineRubrics 👤 Human feedback = noisy & coarse 🧮 Verifiable rewards = too narrow 📋 Static rubrics = rigid, easy to hack, miss emergent behaviors 💡We introduce OnlineRubrics: elicited rubrics that evolve as models train. arxiv.org/abs/2510.07284
live at xAI hackathon!📍 over 500 handpicked devs. only a few hours in & many great projects built already. 1st place winners for each track include an all expensed trip to starship launch. 🚀🚀🚀
🔄RLHF → RLVR → Rubrics → OnlineRubrics 👤 Human feedback = noisy & coarse 🧮 Verifiable rewards = too narrow 📋 Static rubrics = rigid, easy to hack, miss emergent behaviors 💡We introduce OnlineRubrics: elicited rubrics that evolve as models train. arxiv.org/abs/2510.07284
New Scale research: Can smaller models reliably oversee stronger LLM agents? We red team monitoring systems to detect covert sabotage, like agents secretly downloading sensitive information.
Check out 🔥 EgoNormia: a benchmark for physical social norm understanding egonormia.org Can we really trust VLMs to make decisions that align with human norms? 👩⚖️ With EgoNormia, a 1800 ego-centric video 🥽 QA benchmark, we show that this is surprisingly challenging 🤖 🌐 arxiv.org/abs/2502.20490 Our amazing team: MohammadHossein Rezaei* (U of A), Yicheng Fu* , Phil Cuvin* (U of T), @cjziems , @StevenyzZhang , @_Hao_Zhu
.@stanfordnlp papers at @aclmeeting in Vienna next week: • HumT DumT: Measuring and controlling human-like language in LLMs @chengmyra1 @sunnyyuych @jurafsky • Controllable and Reliable Knowledge-Intensive Task Agents with Declarative GenieWorksheets @harshitj__ @ShichengGLiu … @MonicaSLam • Distilling an End-to-End Voice Assistant Without Instruction Training Data @WilliamBarrHeld @StevenyzZhang … @Diyi_Yang • SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs @michaelryan207 @oshaikh13 … @Diyi_Yang • Attacking Vision-Language Computer Agents via Pop-ups @StevenyzZhang … @Diyi_Yang • Mind the Gap: Static and Interactive Evaluations of Large Audio Models @EllaMinzhiLi @WilliamBarrHeld … @Diyi_Yang • Drop Dropout on Single Epoch Language Model Pretraining @houjun_liu @AngledLuffa @chrmanning • ACLED-DS: A Large Multilingual Expert-Annotated Event Dataset for the Real World @sina_semnani … @MonicaSLam • SPHERE: An Evaluation Card for Human-AI Systems @dorazhao9 … @Diyi_Yang • EgoNormia: Benchmarking Physical Social Norm Understanding @mhrezaeics … @Diyi_Yang • Tell, Don’t Show: Leveraging Language Models’ Abstractive Retellings to Model Literary Themes @lucy3_li @camgriffi … @ddemszky
1/🚨 Thrilled to share that our paper (w/ @eduardo_nlp), "Making Language Models Robust Against Negation," has been accepted to the #NAACL2025 main conference! 🎉 #Negation has always been a challenge for language models. Here's our self-supervised method to tackle this issue:
Check out 🔥 EgoNormia: a benchmark for physical social norm understanding egonormia.org Can we really trust VLMs to make decisions that align with human norms? 👩⚖️ With EgoNormia, a 1800 ego-centric video 🥽 QA benchmark, we show that this is surprisingly challenging 🤖 🌐 arxiv.org/abs/2502.20490 Our amazing team: MohammadHossein Rezaei* (U of A), Yicheng Fu* , Phil Cuvin* (U of T), @cjziems , @StevenyzZhang , @_Hao_Zhu
Check out 🔥 EgoNormia: a benchmark for physical social norm understanding egonormia.org Can we really trust VLMs to make decisions that align with human norms? 👩⚖️ With EgoNormia, a 1800 ego-centric video 🥽 QA benchmark, we show that this is surprisingly challenging 🤖 🌐 arxiv.org/abs/2502.20490 Our amazing team: MohammadHossein Rezaei* (U of A), Yicheng Fu* , Phil Cuvin* (U of T), @cjziems , @StevenyzZhang , @_Hao_Zhu
