raghava uppuluri

🎯 underestimate #2 below at your own peril

can someone plz make a head-mounted data capture device designed to be forgotten about while building hardware hardware req: - lightweight - all day life, can have a usb-c port for external charging dont even need battery onboard, can even ship with a toolbelt with the battery even - i dont wear glasses or hats, so ideally doesn't need to be either of those form factors, though id prefer glasses to hats - audio, high quality cameras (include thermal camera for debugging pcbs/motors/robots), high quality enough for detailed playback/posting - speaker (bone conducting ones) - always on recording (dont want to ever miss recording something bc i forgot to press record) - physical clip button - looks like something tony stark would wear software - streams to laptop/phone only, rolling buffer of whatever time i choose - view clips from my laptop/phone instantly whenever i want to - rolling qr code of timestamp so i can sync the camera stream to some hardware experiment im doing just by looking at the screen - can seamlessly integrate into my dev workflow (like wispr) so i can talk to claude code handsfree and ideally have it share context with what im seeing and hearing (make the device firmware itself dead simple and have the algo that determines if im done talking run on the laptop/phone). i'll pay a lot for even a prototype that works.

The first time we rolled a robot into a new warehouse, it didn’t perform as well as we expected. It took us an entire day of debugging, before we realized it was something simple… the cameras were wired completely wrong. 🤦 Left camera to right gripper 🔀 right camera to left gripper. But what was interesting was, the model still kind of worked. In post-training data, left side = bin, right side = conveyor. But even when swapped, the model would still do the task—just slightly worse. Enough to fool us into thinking it was something else for hours… until the moment we switched the cameras back. Then it worked great. This wasn’t the first time we’ve seen emergent ambidexterity. During a packing demo last year, we spotted the robot using the “wrong” hand to shake a USB brick out of a tight baggie. Totally outside the post-training data (we watched all 17 hrs of it to double check). 100% left hand, but for some reason at inference time, it felt the need to use the right hand. Nothing in the model architecture could obviously explain this kind of invariance. If these models are headed where I think they are, imagine one day having a generalist “substrate of intelligence” where you can plug in any number of sensors and actuators, and the whole thing just springs to life. It wouldn’t matter how you wired it up. It would just work. That would be pretty cool.

@sentientcar There are fast ones like @kevin_zakka ‘s mink github.com/kevinzakka/min…

@raghavaupp Yeah though i want to just drop the stl and have the library figure out the collisions + IK. This should be quite a well solved problem why waste time rolling my own or are there no fast libraries ?

Got a few questions about how fast of a UMI gripper motion can it keep up with. Most jumps here are from the IK not being fast enough. Need to understand how to do that faster. Anyone has recs ?

@sentientcar I did dm you with a video of ours :)

@sentientcar Write your own with opus and #diff_ik_w_constraints" target="_blank" rel="nofollow noopener">manipulation.csail.mit.edu/pick.html#diff…

@sentientcar @GeneralistAI Really cool! What’s the top wrist speed your slam can do before losing tracking? Would assume adding imu would help with this. I notice you aren’t rotating the umi in the video, can your slam track well with heavy rotations?

6 dof tracking of umi style hand and replicating it on a robot. @GeneralistAI you better watch out !

Curious why the policy decided to open the bag at 0:52. The bag seems to look empty from the viewpoints of both wristcams. Policies need efficient memory architectures to solve occlusions well. Really amazing demo!

@DaimonRobotics ships

Tribology is where the demons hide

But it’s bad to rely on swapping alone. There is likely materials we haven’t discovered yet or ones under the radar that wear at slower rates where it’s much less of a problem. We must find them and commoditize them with cheap robots. Companies like dmrobot.com/en/ have developed skin that can last millions of cycles under constant wear.

This is GEN-1 putting paper bills into a wallet. Paper has always been deceptively hard for robots. Thin, deformable, and unforgiving—it bends, folds, and slips. Not precise, you miss. Too much force, you crumple. Easy for humans. But for robots, it’s a full-stack challenge.

@JieWang_ZJUI @sincethestudy High quality controls, IK, middleware makes sure your hardware is at the right place at the right time and affects everything from teleoperator cognitive load to hardware lifespan. Data, models, and evals are always downstream.

@sincethestudy Amazing! But I think the core of hardware iteration should always serve for data, models and evaluations, some control stack here could just be 'good to have'?

Join Bracket Bot if you want to experience the fastest iterating full stack hardware team in the world. We are on generation 7 of our robot in just 8 months, generation 8 will launch with a buy now button in 2 months. We write our robot middleware, drivers, inverse kinematics, impedence controller, teleoperation stack, model training and eval pipeline, and much more. literally everything.

just sysid’ed the cambrian explosion