Chen Wang

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

I will join Northwestern University Computer Science as an Assistant Professor in Fall 2026! I am actively recruiting PhD students and seeking collaborations in robotics, human-robot interaction, brain-computer interfaces, cognitive science, societal impact of AI & automation, and AI for art & design. Please see the recruitment announcement on my personal website, and feel free to reach out!

@HaozhiQ @YiMaTweets @JitendraMalikCV Congrats Haozhi!

I’m incredibly honored and thrilled to receive the Lofti A. Zadeh Prize🏆! Huge thanks to the EECS award committee, my advisors @YiMaTweets and @JitendraMalikCV, and all my amazing collaborators. Grateful for the support, mentorship, and inspiration throughout my PhD journey.

🤖 Ever wondered what robots need to truly help humans around the house? 🏡 Introducing 𝗕𝗘𝗛𝗔𝗩𝗜𝗢𝗥 𝗥𝗼𝗯𝗼𝘁 𝗦𝘂𝗶𝘁𝗲 (𝗕𝗥𝗦)—a comprehensive framework for mastering mobile whole-body manipulation across diverse household tasks! 🧹🫧 From taking out the trash to laying out clothes and cleaning toilets—𝗕𝗥𝗦 equips robots to handle practical, everyday activities. 🌐 Explore more: behavior-robot-suite.github.io Let's dive in! 🤿🧵

Sim2Real RL for Vision-Based Dexterous Manipulation on Humanoids toruowo.github.io/recipe/ TLDR - we train a humanoid robot with two multifingered hands to perform a range of dexterous manipulation tasks robust generalization and high performance without human demonstration :D

@agrimgupta92 Congrats Agrim!🤩

Today we are introducing Veo2: SOTA video generation model. You can try the model here: labs.google/fx/tools/video…

A good hand can push intelligence development. Introducing Eyesight Hand, equipped with full-hand high-res tactile sensors and proprioceptive actuators. It is compliant, agile, and powerful. Good tactile sensing makes learning more efficient and robust. Shout out to Branden!

Introducing EgoMimic - just wear a pair of Project Aria @meta_aria smart glasses 👓 to scale up your imitation learning datasets! Check out what our robot can do. A thread below👇

@DJiafei Thanks Jiafei for first introducing AR for data collection! We learned a lot from AR2-D2 and EVE

Really cool to see more work leveraging AR for collecting robot data without a robot! Many times often people neglect that data collection is actually a HRI problem, like how can we make such collection process more ubiquitous and scalable. This is a good demonstration of that along with our work AR2-D2 and recently EVE presenting now at #UIST2024

Excited to introduce ARCap! We found that visual feedback is crucial for high-quality data collection, and AR can greatly help! We invited 20 novice users to each gather a small amount of data using only AR—no robot hardware required. The combined data can successfully train a robot policy!

Why hand-engineer digital twins when digital cousins are free? Check out ACDC: Automated Creation of Digital Cousins 👭 for Robust Policy Learning, accepted at @corl2024! 🎉 📸 Single image -> 🏡 Interactive scene ⏩ Fully automatic (no annotations needed!) 🦾 Robot policies deployed zero-shot in original scene 🌐: digital-cousins.github.io

Synchronize Dual Hands for Physics-Based Dexterous Guitar Playing discuss: huggingface.co/papers/2409.16… We present a novel approach to synthesize dexterous motions for physically simulated hands in tasks that require coordination between the control of two hands with high temporal precision. Instead of directly learning a joint policy to control two hands, our approach performs bimanual control through cooperative learning where each hand is treated as an individual agent. The individual policies for each hand are first trained separately, and then synchronized through latent space manipulation in a centralized environment to serve as a joint policy for two-hand control. By doing so, we avoid directly performing policy learning in the joint state-action space of two hands with higher dimensions, greatly improving the overall training efficiency. We demonstrate the effectiveness of our proposed approach in the challenging guitar-playing task. The virtual guitarist trained by our approach can synthesize motions from unstructured reference data of general guitar-playing practice motions, and accurately play diverse rhythms with complex chord pressing and string picking patterns based on the input guitar tabs that do not exist in the references. Along with this paper, we provide the motion capture data that we collected as the reference for policy training.

@shahdhruv_ @Princeton Congrats!!

Excited to share that I will be joining @Princeton as an Assistant Professor in ECE & Robotics next academic year! 🐯🤖 robo.princeton.edu I am recruiting PhD students for the upcoming admissions cycle. If you are interested in working with me, please consider applying.

@xiaolonw Congrats!

I am deeply honored to receive the 2024 J.K. Aggarwal Prize! It is humbling to follow in the footsteps of such an esteemed group of AI researchers. iapr.org/awards/icpr-aw…

@elliottszwu @Cambridge_Eng @_atewari Congrats Elliott!

I will be joining @Cambridge_Eng as an Assistant Professor in spring 2025, together with @_atewari. Clearly have been missing the good old UK rain after a wonderful year in California. Looking forward to opening this new chapter with brilliant colleagues and students!

We found that the relations between keypoints are a powerful way to represent tasks. What’s more exciting is that these keypoint relations can be formulated as constraint satisfaction problems, allowing us to use off-the-shelf optimization solvers to generate complex robot actions. Check out @wenlong_huang's thread for more on how we automate the keypoint constraints generation process and how this enables robots to perform reactive, bimanual, and long-horizon tasks!

We are hosting the dexterous manipulation workshop at CoRL this year 🤖! We'll dive into topics like visual & tactile perception, skill learning, and control. Don’t miss the opportunity to share your amazing works and participate! dex-manipulation.github.io/corl2024/

Meet our AI-powered robot that’s ready to play table tennis. 🤖🏓 It’s the first agent to achieve amateur human level performance in this sport. Here’s how it works. 🧵

@zhijianliu_ @UCSanDiego @HDSIUCSD huge congrats!

📢 Excited to share that I’ll be joining @UCSanDiego @HDSIUCSD as an Assistant Professor in January 2026! My lab will focus on efficient AI. I'm recruiting PhD students from HDSI/CSE in the Fall 2024 cycle and also looking for RAs/interns! More info see zhijianliu.com.

@shahdhruv_ @svlevine @berkeley_ai @GoogleDeepMind Congrats!

I “defended” my thesis earlier today — super grateful to @svlevine and everyone at @berkeley_ai for their support through the last 5 years! 🐻 Excited to be joining @GoogleDeepMind and continue the quest for bigger, better, smarter robot brains 🤖🧠

Excited to share a new humanoid robot platform we’ve been working on. Berkeley Humanoid is a reliable and low-cost mid-scale research platform for learning-based control. We demonstrate the robot walking on various terrains and dynamic hopping with a simple RL controller.