Jimmy (Tsung-Yen) Yang
46 posts
Jimmy (Tsung-Yen) Yang
@JimmyTYYang1
Researcher at @Waymo Research; ML/Embodied AI; PhD @Princeton ECE
NYC انضم Ağustos 2018
281 يتبع344 المتابعون
It was great to work with @BasileTerv987 on this project! The JEPA world model works well on unseen robot tasks. Thanks Basile for the amazing work!
Basile Terver@BasileTerv987
My first PhD paper is out! 🎓 "What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?" tl:dr: JEPA-WMs for robotics: learn dynamics on top of visual encoders, optimize actions towards goal 👇 w/ @JimmyTYYang1, Jean Ponce, @AdrienBardes, @ylecun
English
It is my pleasure to work with Raktim! A concrete example of how to use human videos to learn robot policies
Raktim Gautam Goswami@raktimgg
Introducing DexWM: Dexterous Manipulation World Model tl;dr: train on human videos; fine-grained actions; hand-consistency loss; DexWM+MPC -> zero-shot dexterous manipulation w/ @_amirbar, @DavidJFan, @JimmyTYYang1, @GaoyueZhou, P. Krishnamurthy, M. Rabbat, F. Khorrami, @ylecun
English
LLMs can't plan or reason, but with help, they can be an important part of systems that do solve really hard reasoning tasks
English
VLMNM workshop starting off strong with this talk by Subbarao Kambhampati of ASU. A really interesting discussion about the way LLMs and VLMs "think" and how it's different. Which leads to...
English
@chris_j_paxton this is so cool! were you able to see the robot doing the demo that they showed in the video?
English
Unitree g1
2-3 kg per hand payload
2 hrs battery life
Much smaller than expected in person
English
It was so much fun to work on this: llama3 + sim2real skills, all are open-sourced!
Dhruv Batra@DhruvBatra_
FAIR researchers (@AIatMeta) presented SegmentAnything and our robotics work at the White House correspondents’ weekend. Llama3 + Sim2Real skills (trained with @ai_habitat) = a robot assistant
English
Jimmy (Tsung-Yen) Yang أُعيد تغريده
FAIR researchers (@AIatMeta) presented SegmentAnything and our robotics work at the White House correspondents’ weekend.
Llama3 + Sim2Real skills (trained with @ai_habitat) = a robot assistant
The Hill@thehill
Washingtonians delved into the world of artificial intelligence (AI) at the Washington AI Network’s inaugural weekend TGAIFriday Lunch for White House correspondents. trib.al/FwHF9Um
English
In case you don't get the reference, Clever Hans was a horse that was claimed to be able to do math, but was really just watching its trainer. In a lot of these LLM examples, the user does most of the hard work of verifying the solution.
Find me on bsky @colin-fraser.net@colin_fraser
When you add stuff like this when it gets it wrong you’re just doing a clever hans thing. The way to see that is to try adding the same thing when it gets it right.
English
@chichengcc very cool! How long does it take to collect these demo data?
English
This model is far from perfect. For example, it doesn't work well under direct sunlight since it constantly rained at Stanford during our data collection effort. Please share your failure cases! Hopefully, we can have a community-based effort to train an even more robust model!
English
Weights drop ⚠️
We released our pre-trained model for the cup arrangement task trained on 1400 demos! We aim to enable anyone to deploy UMI on their robot to arrange any "espresso cup with saucer" they buy on Amazon.
github.com/real-stanford/…
English
@PeterMitrano @MarkVanderMerwe very cool demo! We are also building some infra for Spot robot: github.com/facebookresear…
Using this could speed up image collection!
English
Raw video of us collecting data for behavior cloning. Not flashy, but honestly pretty fast & efficient. @MarkVanderMerwe helping with cable management :)
English
Very excited to see that Habitat 3.0 is finally out! Please also check out our paper here: ai.meta.com/static-resourc…
Dhruv Batra@DhruvBatra_
Announcing Habitat 3.0, simulating humanoid avatars and robots collaborating! - Humanoid sim: diverse skinned avatars - Human-in-the-loop control: mouse/keyboard or VR - Tasks: social navigation and rearrangement Over 1,000 steps per second on 1 GPU for large-scale learning!
English
Română
Please check out our Spot Sim2Real repo! github.com/facebookresear…
It aims to reproduce the demos of ASC and LSC. We open-source the weights of the navigation, picking, and place skills, and the infrastructure to use Large Language Models (LLMs) to interact with those skills!
AI Habitat@ai_habitat
Habitat stable version v0.2.5 released! -- 🤖 Hardware bridge for BD Spot -- 📷 Improved PBR rendering for better synthetic scene visuals -- 🤗 ai-habitat on HuggingFace -- 🏠 Habitat Synthetic Scenes Dataset (HSSD) and more... github.com/facebookresear… github.com/facebookresear…
English
@chris_j_paxton I saw this too!!! It will be big and transformative if this is true! Look forward to seeing other researchers to verify this
English
Recent claims about manufacturing room temperature superconductors. This is the definition of "big if true"
A bit sad to be a cynic but I can't imagine it is. Hope I'm wrong
Prakash@8teAPi
Real Superconductors Might Be Here - we won’t know for sure for a couple of years - Wikipedia page is a litany of hoaxes - current claim is full on room temperature and pressure full superconductor manufactured from lead. - superconducting because of its structure - if true would be another Industrial Revolution Super annoying that you can’t really know whether it’s real for a loong time. But it’s in Korea and they have all the heavy industry that can deploy quickly. arxiv.org/pdf/2307.12008…
English
Jimmy (Tsung-Yen) Yang أُعيد تغريده
“We were very interested in testing out high-level reasoning and planning. We wanted to see...if the robot could figure out the steps it needed to take to solve the problem.” - Discover why @MetaAI chose Spot for their research. bit.ly/44ILlhG
English
This is our recent work building toward "reproducible" robotic research. Thanks @chris_j_paxton for leading such an amazing project!
Chris Paxton@chris_j_paxton
The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀 aihabitat.org/challenge/2023…
English
Check our work at Meta AI #CVPR23 ! Spot is able to understand natural language instructions and complete the task for you!
AI at Meta@AIatMeta
We’ll also be showcasing a handful of demos on the showfloor. We brought Spotty to demo some of our exciting recent work towards general-purpose embodied AI agents. More on this research ➡️ bit.ly/3N8LeVg 5/6
English