SceniX AI

𝗧𝗲𝘀𝘁𝗶𝗻𝗴 𝗿𝗼𝗯𝗼𝘁 𝗽𝗼𝗹𝗶𝗰𝗶𝗲𝘀 𝗶𝗻 𝘁𝗵𝗲 𝗿𝗲𝗮𝗹 𝘄𝗼𝗿𝗹𝗱 𝗶𝘀 𝗲𝘅𝗽𝗲𝗻𝘀𝗶𝘃𝗲, 𝘀𝗹𝗼𝘄, 𝗮𝗻𝗱 𝗵𝗮𝗿𝗱 𝘁𝗼 𝗿𝗲𝗽𝗿𝗼𝗱𝘂𝗰𝗲. 𝗕𝘂𝘁 @Columbia 𝗮𝗻𝗱 @SceniXai 𝗷𝘂𝘀𝘁 𝗯𝘂𝗶𝗹𝘁 𝗮 𝘀𝗶𝗺𝘂𝗹𝗮𝘁𝗼𝗿 𝘁𝗵𝗮𝘁 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝘄𝗼𝗿𝗸𝘀. They reconstruct real environments as soft-body digital twins using Gaussian Splatting for photorealistic rendering. Then they evaluate robot policies in simulation, and the results correlate strongly with real-world execution. They then tested it on tough tasks like plush toy packing, rope routing, and deformable object manipulation, the kind of stuff that’s notoriously difficult to simulate. 𝗪𝗵𝗮𝘁 𝗺𝗮𝗸𝗲𝘀 𝗶𝘁 𝘄𝗼𝗿𝗸 𝗶𝘀 𝗽𝗵𝘆𝘀𝗶𝗰𝘀 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻. They tune the digital twin’s parameters to match real-world dynamics, not just visuals. Plus, color alignment helps close the appearance gap between simulated renderings and real camera feeds. The results show that simulated rollouts predict real performance across multiple state-of-the-art imitation learning policies. Once you have this infrastructure, you can run hundreds of policy evaluations overnight. Evaluation has always been one of the biggest bottlenecks in robot learning and this is exactly the kind of infrastructure that can unlock it. Check out the paper here: real-to-sim.github.io Video credit: @SceniXai

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐 real2sim-eval.github.io

🧵 Evaluating robot policies in the real world is slow, expensive, and hard to scale. During my internship at @SceniXai this summer, we had many discussions around the two key questions: how accurate must a simulator be for evaluation to be meaningful, and how do we get there? Our new framework, Real2Sim-Eval, takes a step toward that answer. By combining Gaussian Splatting for photorealistic rendering and soft-body digital twins for realistic dynamics, we make simulation predictive of real-world performance. 👉 real2sim-eval.github.io