Hanyang Chen

@nanjiang_cs 4 hours ago it’s theoretical online RL and now testing the theory

quack quack

🤖 Feeling excited about the future of household robotic agents (i.e., embodied agents)? You should also consider their safety! 🔪Meet BEAT: the first visual backdoor attack on MLLM-based embodied agents. 🧵 1/7

Its truly touching to see the solidarity of academic community in this wave of meta layoff

World Model Reasoning for VLM Agents (NeurIPS 2025, Score 5544) We release VAGEN to teach VLMs to build internal world models via visual state reasoning: - StateEstimation: what is the current state? - TransitionModeling: what is next? MDP → POMDP shift to handle the partial observability from visual states! mll.lab.northwestern.edu/VAGEN/ 🙌Led by @James_KKW @WilliamZhangNU @wzihanw @yaning_gao @LINJIEFUN @qineng_wang @hc81Jeremy @w4nanch1 @2prime_PKU @zhengyuan_yang lijuanwang,@RanjayKrishna @jiajunwu_cs @drfeifei @YejinChoinka 👍Grateful for the joint effort of @northwesterncs @uwcse @StanfordAILab @microsoft @WisconsinCS @siebelschool.

Sincere appreciation to @ZihaoZH94437841 @RuiYang70669025 @EmpathYang @ExplainMiracles @James_KKW @jackbai_jkb @zhenhailongW, @rui4research ,Prof. ChengXiang Zhai(@TIMANUIUC) @hengjinlp @uiuc_nlp @ManlingLi_ @huan_zhang12 and Prof Tong Zhang. They made it possible.

🧵 Thread 4 🎬 Case Study After ERA training, a 3B model that originally failed every task can now reason and act step-by-step: (a) On EB-ALFRED, it reflects on earlier mistakes to finally clean and place the plate correctly. (b) On EB-Manipulation, it accurately fits the star into the right slot. 💭 Small models can just see clearly, think deeply, and act precisely as giants, but more efficient.

🤖️Today we introduce the Embodied Reasoning Agent (ERA), a framework that transforms a compact Vision Language Model (VLM) into a performant and efficient embodied agent. When large models like GPT-4o and Gemini show strong embodied performance on EmbodiedBench, smaller ones often fail completely. But, if a robot can’t fit a huge model, how can a small one still understand the world, plan tasks, and act precisely? 💡 That brings out ERA — By asking 1. What prior knowledge does embodied agent require before RL? and 2. What make RL in long-horizon embodied task stable and effective? We distill them into a unified post-training regime that is capable of delivering both high-level planning agent and low-level control agent, by different curation of training data. 🔗 Web link: embodied-reasoning-agent.github.io 📄 Pape link: arxiv.org/abs/2510.12693 🧵 Thread 1）

@ManlingLi_ Appreciation for the continuous support, Manling.

Check @hc81Jeremy ’s oral presentation on EmbodiedBench at ICML! He is also applying to PhD this year, please reach out to him at ICML!

Grateful for the chance to present EmbodiedBench at ICML as an Oral. A rewarding experience full of learning. Thanks for @RuiYang70669025 @hengjinlp @jyzhang1208 @huan_zhang12 Mark_Zhao @ManlingLi_ Tong_Zhang and many others who make it possible. See you next time.

Thanks Prof. Yu to be at our talk @Zhou_Yu_AI. Happy ICML25🇨🇦

@RuiYang70669025 @hc81Jeremy Good work, enjoyed the talk.

My coauthor @hc81Jeremy will present EmbodiedBench at ICML 2025! 🤖 Oral Session 6A 📍 West Hall C 🕧July 17 3:30-3:45 pmPDT 📌 Poster Session 📍 East Hall A-B #E-2411🕜 July 17 4:30-7 pm PDT Come say hi and let’s talk about VLM agent training, evaluation, and benchmarking! 😀

I'll be at ICML next week, presenting our paper on Wasserstein Policy Optimization on Tuesday! If you're in Vancouver, come say hi!

Excited to share that EmbodiedBench was selected for an Oral at ICML 2025! We recently added results for new models (InternVL3, Gemma3, Ovis2) and released a large agent trajectory dataset on 🤗: embodiedbench.github.io Try training and evaluating your MLLM for embodied agent!

How to schedule a meeting? When you ask for a meeting with others, you are asking for their time. You are asking for their most valuable, finite resource to benefit yourself (e.g., for advice, networking, questions, and opportunities). Here are some tips that I found useful.

🚀 Excited to share our latest work on Iterative-DPO for math reasoning! Inspired by DeepSeek-R1 & rule-based PPO, we trained Qwen2.5-MATH-7B on Numina-Math prompts. Our model achieves 47.0% pass@1 on AIME24, MATH500, AMC, Minerva-Math, OlympiadBench—outperforming LLaMA-3.1-70B-Instruct and approaching Eurus-2-7B-PRIME! With SFT warm-up + Iterative-DPO, we reached 51.8%, surpassing Qwen2.5-7B-SimpleRL-Zero and matching our PPO-Zero. 🔍 Key takeaways: 1️⃣ NLL loss doesn't help DPO. 2️⃣ DPO on MATH alone saturates—diverse prompts (Numina-Math) help. 3️⃣ RAFT (Rejection Sampling Finetuning) is simple & effective for rule-based RL. 4️⃣ Long CoT for warm-up SFT significantly improves DPO, reaching PPO-level performance. 5️⃣ Base models already show self-reflection ("aha moments"), RL doesn’t boost this. 6️⃣ Response length changes are inconsistent, no upward trend. 7️⃣ PPO still wins—DPO & RAFT improve models but fall slightly short of PPO (51.8%). 📜 Code & models all open-sourced! Try them out & share feedback! 🔗 GitHub: github.com/RLHFlow/Online… 🔗 Notion: efficient-unicorn-451.notion.site/Online-DPO-R1-…

Thanks, Manling, for sharing the work! Improve your VLM with EmbodiedBench.

🔥Exploring MLLM as Embodied Generalist. 🔍 4 diverse tasks - from High Level Planning to Low Level Manipulation 🎯 6 fine-grained evaluation capabilities ALL IN ONE MLLM. 📊 More than a Benchmark - A standardized platform for more algorithms to sparks.