Zeon Systems

AI is writing papers. Scientists are still loading centrifuges. We have built a perception, control, and manipulation stack that lets robots run experiments flexibly in scientific labs. We exist to expand the boundary of what is possible to automate today. Over the next week, we’ll share deeper dives into how our world models, mesh systems, motion planning, and LLM agents come together to make it possible. Centrifuges are everywhere in science - and a nightmare for automation. Here’s a quick preview of our system loading tubes into one.

@minhash say no more!

@zeonsystems I want the margarita skill so bad

Agents can now run real lab workflows. We taught a robot to vortex a rack of tubes in minutes - from natural language. Flexible to write, deterministic to run. build modular skills → compose workflows → simulate → deploy Here's how it works🧵

1/ Write a skill - a reusable robot capability built from low-level primitives that an agent or human can use. Think "pick up an object" or "open a machine" 2/ Compose skills into a workflow. Preview it in simulation before anything touches real hardware. 3/ When it looks right, run it. Our live localization means the workflow works even if objects move - no re-programming if your vortexer shifts position. Engineers/Agents keep full control. Skills, workflows, and world models are all based on Python. Inspect them, modify them, reuse them across experiments. You build a library, not a pile of one-off scripts.

MOTION PLANNING Because the simulation stays synchronized with reality, robots can safely plan complex motions around real equipment and objects. Dodging shelves, hopping over machines, avoiding each other. Both arms reason about the same live scene and can move simultaneously for higher throughput. This enables coordinated manipulation of lab equipment - opening machines, moving plates, handling tubes, etc.

TRACKING SCIENTIFIC STATE Workflows depend on knowing where samples are and what has happened to them. Tubes, plates, and reagents are persistent entities in the system. When they move between instruments or their contents change, their state updates in the simulated world. This gives the system context on the current state of the lab, allowing workflows to execute reliably. There’s still a long way to go in capturing the full scientific state, but this layer is critical for lab robotics.

Robots fail in labs for a simple reason: the world keeps changing. Previously we showed how we build world models of scientific labs from scans of the real environment. But a one-time snapshot isn’t enough. In real labs, machines change state (open/closed, loaded/unloaded), tubes and plates move between instruments, and samples transfer between containers. So we maintain a live representation of the lab - continuously updating both the geometry and the scientific state of the system. This lets robots operate reliably even as the lab changes around them. It enables three core capabilities: 1) Tracking physical state – moving machine components (like lids or doors) are continuously localized. 2) Tracking scientific state – tubes, plates, and samples persist as entities whose state updates as the robots run. 3) Motion planning – robots plan safe dual-arm actions directly in the live world model. Check these all out below 🧵

Simulated lab worlds straight from pixels. We scan the lab scene with depth cameras and reconstruct the environment. Each object already has a high-quality mesh in our object library from our mesh scanning system. A localization pipeline places those meshes into a shared coordinate frame, creating a digital version of the lab. This works well for science automation because the environment is constrained: the set of objects is limited, we have accurate models of them, and we don’t need to solve the full open-world perception problem. With this digital lab in place we can: • Drive safe motion planning around real geometry • Track state changes by re-localizing mesh parts (e.g. a lid opening) • Attach objects to the robot when picked so the planner accounts for them and avoids collisions built by @BastotdeHeijden @BlerimAbdullai @TahirMello

You need sub-millimeter precision to automate scientific labs. Calibration is crucial to achieving this. It resolves exactly where cameras and robot arms are with respect to the world and to each other. We have put a lot of work into robust camera and arm calibration so our system understands its environment precisely. Below we show a simple task: inserting a microcentrifuge tube into a tube rack. With proper calibration, inserting into these holes goes from impossible to achievable. - cc @BlerimAbdullai and @BastotdeHeijden for leading this work

Robots should be able to use any machine in a lab. To do that, robots need to reason about objects: their geometry, how they move, and how to avoid colliding with them. Our solution is meshes: 3D representations of real-world objects. We create these by leveraging depth information from cameras. With a mesh-based representation, we can explicitly model geometry, plan safe motions, and maintain precise alignment independent of lighting conditions or visual noise. We also learned the hard way that RGB data on it’s own (without depth) is not enough after several attempts to operate purely in pixel space. We built simple tools that let anyone: • Create meshes - scan objects with depth cameras or import existing models • Annotate meshes - mark joints, buttons, lids, and other moving parts • Build a shared mesh database - a growing library of annotated objects used by the system A new object can be scanned, annotated, and ready for the robot in no time. Below is an example of scanning and annotating a centrifuge mesh. cc @BastotdeHeijden @BlerimAbdullai for spearheading this work

@ATinyGreenCell no way - divide that by 4. these arms are <$10k each (linked below). we keep hardware costs low so scientists can afford them. ufactory.us/product/ufacto…

@zeonsystems Those arms cost like 80k right?

AI is writing papers. Scientists are still loading centrifuges. We have built a perception, control, and manipulation stack that lets robots run experiments flexibly in scientific labs. We exist to expand the boundary of what is possible to automate today. Over the next week, we’ll share deeper dives into how our world models, mesh systems, motion planning, and LLM agents come together to make it possible. Centrifuges are everywhere in science - and a nightmare for automation. Here’s a quick preview of our system loading tubes into one.

@chr1sa Let's do it! Will DM you

@zeonsystems I've been deep diving in your space for the past few months, meeting almost everyone, but not yet you! Time to rectify that error? I'm Bay Area based and happy to swing by your offices so we can compare notes.

@chr1sa thank you, what an honor!

@zeonsystems As the author of The Long Tail (and a fan of what you're doing), I applaud this message!

Check out the end-to-end run here: youtube.com/watch?v=KogFyO…

The big improvement is going to come from api-enabled automated labs more than the particular AI running them

one of the coolest engineers I know has shipped us a beautiful piece of hardware check out @JannikGrothusen at the @RobotLearningCo