Ryan Punamiya

351 posts

Ryan Punamiya banner
Ryan Punamiya

Ryan Punamiya

@ryan_punamiya

research intern NVIDIA GEAR Lab | @GeorgiaTech | robotics researcher advised by @danfei_xu @judyfhoffman | all views are my own

Atlanta, GA Entrou em Kasım 2017
314 Seguindo547 Seguidores
Tweet fixado
Ryan Punamiya
Ryan Punamiya@ryan_punamiya·
EgoVerse embodies the future of collaborative research, not just for egocentric human data. It's a great time for academia and industry to leverage synergistic collaborations to push studies to a scale where it matters. 🚀 We would love to get in touch with other labs interested. So please reach out to me, @simar_kareer, @danfei_xu or many of our wonderful collaborators in the consortium.
Danfei Xu@danfei_xu

Introducing EgoVerse: an ecosystem for robot learning from egocentric human data. Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling 1300+ hrs, 240 scenes, 2000+ tasks, and growing Dataset design, findings, and ecosystem 🧵

English
2
3
25
4.3K
Ryan Punamiya retweetou
Danfei Xu
Danfei Xu@danfei_xu·
Gave a talk on Robot Learning from Human Data at Stanford. It was great to be back! Some opinionated points: 1. Human data collection capacity is outpacing the research. 2. We still don't have the "science" for scaling robot capability with human data. 3. We are far from being able to model naturalistic human behaviors. youtube.com/watch?v=NUtaN1…
YouTube video
YouTube
Danfei Xu tweet media
English
1
21
179
13.6K
Ryan Punamiya
Ryan Punamiya@ryan_punamiya·
EgoVerse embodies the future of collaborative research, not just for egocentric human data. It's a great time for academia and industry to leverage synergistic collaborations to push studies to a scale where it matters. 🚀 We would love to get in touch with other labs interested. So please reach out to me, @simar_kareer, @danfei_xu or many of our wonderful collaborators in the consortium.
Danfei Xu@danfei_xu

Introducing EgoVerse: an ecosystem for robot learning from egocentric human data. Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling 1300+ hrs, 240 scenes, 2000+ tasks, and growing Dataset design, findings, and ecosystem 🧵

English
2
3
25
4.3K
Ryan Punamiya retweetou
Project Aria @Meta
Project Aria @Meta@meta_aria·
⚡️EgoVerse is a first-of-its-kind, collaborative ecosystem for human-to-robot learning. The consortium leverages Project Aria to capture high-fidelity, egocentric human data — including 3D hand and head poses — to train next-gen robot manipulation policies. With over 1,300 hours of data across 2,000+ tasks, EgoVerse is a prime example of how the Aria Research Kit is being used by our partners to accelerate the future of embodied AI. Learn more: 🔗egoverse.ai 📰 arxiv.org/abs/2604.07607 Apply for the Aria Research Kit: projectaria.com/research-kit/ #MachineLearning #Robotics #ProjectAria #EgoVerse #ComputerVision @simar_kareer , @ryan_punamiya , @RogerQiu_42, @XiongyiCai , @yexelal
English
2
44
226
17.3K
Ryan Punamiya
Ryan Punamiya@ryan_punamiya·
This is an awesome work! Glad to see there is further study on structured representation alignment and some more theoretical study that validates and expands the results we found in EgoBridge
Yu Lei@_OutofMemory_

🤖Co-training is everywhere (sim↔real[e.g. GR00T, LBM], human↔robot[e.g. PI, EgoScale], even non-robot data[e.g. PI, LBM). But why does it work? How can we improve it further? Taking sim-and-real imitation learning in diffusion/ flow-based models as the test bed, we performed a rigorous mechanistic analysis, drawing on theoretical insights and multi-layered experiments. 😮Key insight: it’s all about representations. - Alignment → enables transfer - Discernibility → enables adaptation ⚖️Both are necessary — it's better to have more aligned representations, but the model must be able to discern the domains. We term this as structured representation alignment. ⬇️Let’s take a deep dive into that: Paper: arxiv.org/pdf/2604.13645 Website: science-of-co-training.github.io

English
1
3
24
4.6K
Ryan Punamiya retweetou
Kye Gomez (swarms)
Kye Gomez (swarms)@KyeGomezB·
Introducing OpenMythos An open-source, first-principles theoretical reconstruction of Claude Mythos, implemented in PyTorch. The architecture instantiates a looped transformer with a Mixture-of-Experts (MoE) routing mechanism, enabling iterative depth via weight sharing and conditional computation across experts. My implementation explores the hypothesis that recursive application of a fixed parameterized block, coupled with sparse expert activation, can yield improved efficiency–performance tradeoffs and emergent multi-step reasoning. Learn more ⬇️🧵
Kye Gomez (swarms) tweet media
English
236
1.2K
8.2K
1.6M
Ryan Punamiya
Ryan Punamiya@ryan_punamiya·
wow! emergent cross embodiment transfer and pre-trained better than specialist! And monumental engineering effort to add world model, HL, and VLA
Physical Intelligence@physical_int

Our newest model, π0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!

English
0
0
23
1.8K
kache
kache@yacineMTB·
My biggest regret is getting a computer science degree instead of a mechanical engineering degree. I can't believe I spent 4 years getting a degree in "use hash table for o(1) look ups"
English
228
171
6.2K
446.2K
Ryan Punamiya retweetou
Chris Paxton
Chris Paxton@chris_j_paxton·
The best robotics videos are always a montage like this -- its the only way to show the power and generality of your approach. SONIC from NVIDIA is a huge step towards a general purpose robot control model. Learn more about it on our podcast ->
RoboPapers@RoboPapers

How can we build a general-purpose “foundation model” for robot motion? @zhengyiluo joins us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a model with a generally-useful embedding space that can be controlled by a VLA, or from human video, to perform a wide variety of humanoid whole-body-control tasks, including with zero-shot transfer to previously unseen motions. Watch episode 72 of RoboPapers, with @micoolcho and @DJiafei, now!

English
5
8
131
15.4K
Keerthana Gopalakrishnan
Keerthana Gopalakrishnan@keerthanpg·
There are really few (< 15) people in the world who know both frontier modeling AND modern robotics very well. A lot of strong roboticists are still working off of ideas from the pre-2023, pre-Gemini era of robotics and know very little about frontier AI techniques. A lot of strong frontier modeling people do not care yet / have little expertise in robotics. The latter group is increasing as more of the big AI labs are foraying into robotics but still the world needs a lot more :)
English
54
47
694
93.9K
Ryan Punamiya
Ryan Punamiya@ryan_punamiya·
in sf tomm, if anyone wants to have coffee chats
English
0
0
3
412
Ryan Punamiya retweetou
Dhruv Shah
Dhruv Shah@shahdhruv_·
If you haven't already, check out LAP: a cross-embodiment VLA that is fully open and works on most single-arm robots out-of-box! lap-vla.github.io LAP is now #2 on the MolmoSpaces: the top fully open policy model and it was fully trained on lab compute @EPrinceton 🐯🚀
Dhruv Shah tweet media
Lihan Zha@LihanZha

LAP now ranks 2nd on the Molmospace leaderboard @allen_ai, and is the only model that is: 1. Fully open-sourced (data, checkpoint, code) 2. Evaluated out-of-the-box, even _without_ fine-tuning on DROID! 3. From academia molmospaces.allen.ai/leaderboard

English
4
11
86
11.7K
Ryan Punamiya retweetou
Younghyo Park
Younghyo Park@younghyo_park·
What's different between these two BC policies? It's the same architecture, training budget, and data collection setup — the only difference is the controller gains! Controller gains are an understudied design parameter in robot learning. In our new work (w/ @BronarsToni*, @pulkitology), we show how they act as an inductive bias across BC, RL, and Sim2Real transfer, with real consequences on performance. Here's what we found 🧵 * Equal Contribution 📄arxiv: arxiv.org/abs/2604.02523 🔗website: younghyopark.me/tune-to-learn/
English
7
78
480
149.9K
Ryan Punamiya
Ryan Punamiya@ryan_punamiya·
@keerthanpg training from scratch doesn’t mean you don’t “inherit from an internet of knowledge” Fwiw, it is still possible to do that by adding auxiliary training objectives of language etc.
English
0
0
4
457
Keerthana Gopalakrishnan
Keerthana Gopalakrishnan@keerthanpg·
This is an amazing blogpost but the conclusions that fall out of it are quite grim. 1. If pre-training from scratch is required, then robotic capabilities will scale as a function of operations expense - linear with respect to number of people employed, number of collect hardware - etc. This leaves most American companies in a hard place as China is better positioned to scale ops due to large scale manufacturing and cheap labor. 2. That we cannot bootstrap off of priors from AI models like VLMs and World Models foreshadows a grim foretelling for the whole field of robotics. If we cannot re-use the internet, leverage the collective human experience already accumulated, we are cooked. Collecting the entirety of human experience on hardware again via tele-operators will take many years if not decades and this would mean physical AI will scale far too slowly. Breakthroughs that allow us to use human data / YouTube / RL/ models like Gemini and Veo - that scales capabilities as a function of FLOPs, data tokens(even if not from robot) - will then become very valuable to break us out of a slow scaling paradigm.
Pete Florence@peteflorence

x.com/i/article/2041…

English
14
18
219
41K
Ryan Punamiya retweetou
Nicholas Roberts
Nicholas Roberts@nick11roberts·
That new LFM2.5-350M is super overtrained, right? And everyone was shocked about how far they pushed it? As it turns out, we have a brand new scaling law for that! 🧵 [1/n]
Nicholas Roberts tweet media
English
11
53
361
67.1K
Ryan Punamiya
Ryan Punamiya@ryan_punamiya·
@peteflorence i’m really curious if this is still a 7B model to do reactive and smooth execution 👀
English
0
0
0
28
Pete Florence
Pete Florence@peteflorence·
Today, we announce GEN-1 and tell more of our story. It is truly an amazing time to be alive. The level of creativity in intelligence we are seeing is crossing into new levels. Not every task we try can be mastered today. But many can, and we now have multiple cases where the models are coming up with entirely new strategies to solve tasks. These improvisations feel more like "ideas", rather than slight adjustments. As one example, we did a task shoving plushy toys into bags on a conveyor belt. In finetuning the model, it was trained to use one hand to open the bag, the other hand to shove in the plushy – nothing fancy. But once when testing the model, the plushy didn't make it all the way in, and the model had the idea to simply pick up the bag with both hands, and shake it so the plushy settled down into the bag. In testing other skills too, we've seen similar emergent strategies where the model coordinates both hands to solve tasks – see the examples with the metal washers in the GEN-1 videos. For language models, it was these glimmers of creativity that lit the spark for many in the GPT-3 era. We are very excited about what's ahead. Amazing work by the whole Generalist team on this model.
Generalist@GeneralistAI

Introducing GEN-1. Our latest milestone in scaling robot learning. We believe it to be the first general-purpose AI model to master simple physical tasks. 99% success rates, 3x faster speeds, adapts in real time to unexpected scenarios, w/ only 1 hour of robot data. More🧵👇

English
19
16
201
23.6K
Ryan Punamiya
Ryan Punamiya@ryan_punamiya·
i think pepsi is actually better than coke
English
1
0
4
403

Descobrir

@eddybuild @simar_kareer @danfei_xu @RogerQiu_42 @XiongyiCai @yexelal @yacineMTB @dwarkesh_sp