PRB

Task progression tracking does wonders! Here are 6 autonomous napkin folds that a bimanual robot did - it could have done way more but i had to go. Some outstanding things: - the data was collected elsewhere - the lighting isnt sufficient and doesnt resemble training data. at moments, there is shadow from the robots EE when it is making precise picks - this is stock hardware. Nothing was modified - the grippers can barely slide underneath the napkins

@ChongZitaZhang Will try. We are on Limx now!

@builds_robots btw training on humanoid is probly much cheaper and faster! I also wrote a guide that might be helpful for an AI agent: zita-ch.github.io/tech-blog/?pos…

How training progresses in multiple distinctive terrains in AME-2 (25% training budget):

github.com/zita-ch/techbl… Since there have been many communications on AME series with other researchers, I wrote a guide for reimplementing AME-1 and AME-2. The AME-2 part is not complete, and I sadly found some people are faster than me (as the author...)

@ChongZitaZhang 🤣🤣🤣 You are the best haha!

@ChongZitaZhang Helps a lot. Thanks!

@builds_robots a new run for anymal

@ChongZitaZhang Ahh awesome will try. Do you mind sharing all your wandb graphs here? Would love to see how off we are

@builds_robots Now the terrain level goes up much faster

@ChongZitaZhang Awesome, happy to contribute as a team if thats valuable!

@builds_robots The plan exists but the progress is slow:(

@ChongZitaZhang Sadly no one in our team has much experience in sim :)

@ChongZitaZhang 🤣 any plans on sharing yours? We have made progress transferring human data to robots and want to integrate AME2 for WBC

@natashamalpani its a mix of all three (with more weight on 2 and 3) + RL (which you mention implicitly) that will solve manipulation and dexterity this year

there is no hugging face for robotics data. no standardized pipeline for collecting, labeling, versioning, training on real-world robot data at scale. no tooling that handles contact dynamics and material deformation well enough for industrial manipulation. no teleoperation infrastructure where human supervisor intervention automatically becomes training data. no vertical-specific manipulation datasets for any specific industrial task. the actual bottleneck in physical AI is the data and the infrastructure to generate it. and this is a structural problem. for language AI, training data was the internet. abundant, cheap, already labeled by human intent. for robotics, the gap between where foundation models are and where they need to be cannot be closed by deploying more robots. three bets are being made right now: simulation-first works brilliantly for locomotion. domain randomization has essentially solved quadruped walking in unstructured terrain. but it breaks down completely for manipulation. simulated cameras have no noise, blur, or friction error. real cameras and grippers have all of it. cable insertion, fabric folding, dexterous assembly are exactly where simulation fails. teleoperation as data collection is the second move. deploy semi-autonomous robots, capture human-guided trajectories, iterate. theoretically sound. but the capital math is brutal and the execution evidence isn't there yet. human video as proxy is the third. if robots could learn from watching humans, you tap unlimited data. the problem: human hand geometry and force feedback don't map onto robot actuators. you're learning the shape of motion without the physics that make it work. what's actually working today is locomotion. narrow manipulation in structured environments. inspection and sensing. quadrupeds doing thermal inspection. no general-purpose manipulation required. the hardware race is loud, capital-intensive, winner-take-few. but the data infrastructure race is quiet, undercapitalized, wide open.

@davefontenot @Kazanjy Also @menemazarakis tho he has a fund now

@davefontenot Zack nathan backed some top robotics startups early And @Kazanjy is the best

Who are the highest signal angel investors in the valley nowadays? (who dont secretly have a fund behind their investing)

@PSYONICinc Do you make gloves and hands separate?

The Ability Hand is now one of the first commercial dexterous hands officially integrated into NVIDIA Isaac Lab. True dexterity data for AI starts with humans. One hand, any robot. Read the announcement: psyonic.io/news/press-rel… #PSYONIC #NVIDIA #GTC #IsaacLab #Robotics

@orcahand awesome work! congrats

@orcahand is a direct drive hands in the works as well?

it's time to drop three new #opensource robotic hands! this time with tactile sensors! Tweak it, 3D print it, and use them in your robotics and physical AI research! Here are some wild examples ↓↓↓

@jackvial89 got it. the answer is yes! dm-ed

i should clarify what I mean by generalization, I mean in a pretty narrow sense as in with a pick a place task with say 50 samples does the reward reinforcement on 20 additional samples 10 success, 10 failure help the model generalize to more than a single position. thank you! if you can share the value network and advantage code that would be very useful to compare against

i'm working on an implementation of π*0.6 RECAP. going to start with a simplified version of the full pipeline

@jackvial89 We used pi0.5 so didnt pretrain with RECAP. It definitely makes it more robust than pi0.5 - we ran one full iteration. I am not sure you should expect generalization. I can DM the value network and advantage calculation code with you.

@builds_robots cool! where you able to get to good generalization with it?

@chris_j_paxton Probably sim when it gets better. Imagine this on Atlas full body rotation 😂

This is a beautiful hand. But how do you collect demonstrations that can utilize it

@RTinkslinger Why do you think few companies will make foundation models? Policies are incredibly small and far cheaper to train compared to ultra-large large LLMs. If teams can crack ego data which is cheap, foundation models are within reach.

Every secondary research regurgitating VC who has never been close to deep learning is going to get their ’physical AI thesis wrong (remember the morons tooting the horn on co-pilots). Rhoda is just the first few to show the path (similar to pre trained BERT cos of 2019). Few cos shall make the foundation model + harness. Rest in venture shall apply and do vertical implementation and win #always the arc.