Frank Dellaert

My Annual Reviews article on Factor Graphs in Robotics is finally out with a publicly accessible link: annualreviews.org/eprint/85PQDQY…

Working on the SLAM Handbook has been one of the highlights of my career — I’m grateful to have collaborated with such an incredible group of co-editors and contributors. The handbook is now free and open-source: dspace.mit.edu/handle/1721.1/… [1/n]

A 3D view of the smoother output:

After introducing the filter itself, we will show how factor graphs can be leveraged for increasingly more sophisticated legged estimators: first a local graph update, thena fixed-lag smoother that estimates bias.

As an application of the invariant EKF in GTSAM, we added a state estimator for legged robots, along with some more factor-graph-heavy variants. Try it on your rosbags - especially interested in validating it with non-quadrupeds (bipeds, hexapods, spiders). It takes IMU and feet in body frame as inputs.

New research from Tsinghua, Peking University and other top labs taught a humanoid robot to play tennis using scattered human movement clips instead of perfect match data. The big deal here is how the team solved the data problem for physical robots. Usually, teaching a robot to do something highly athletic like playing tennis requires perfect, continuous tracking data of professional human players. Getting that kind of flawless 3D physical data during a high-speed match is extremely difficult and expensive. This paper bypasses that massive hurdle entirely. Instead of needing perfect full-match data, the researchers just used short, disconnected, and imperfect clips of basic human swings. The AI system uses these rough clips as a basic hint for how a swing should look, and then a physics simulator corrects the physical errors so the robot does not fall over while swinging to hit the ball. Because they proved they can take messy, fragmented human data and turn it into a smooth, highly dynamic robot athlete, this means we can start teaching robots all sorts of complex physical tasks without needing to record perfect human demonstrations first. It severely lowers the barrier to making robots useful in fast, unpredictable physical environments. The robot successfully tracked fast incoming balls and consistently hit them back to specific target zones while looking surprisingly natural.

GTSAM now has a small hierarchy of Kalman filters for states that live on manifolds and Lie groups: ManifoldEKF -> LieGroupEKF -> InvariantEKF -> LeftLinearEKF The key idea is simple: keep the state on the manifold, but do uncertainty propagation in the tangent space. For navigation and IMU integration, this matters a lot.

We also have the machinery for equivariant filters (pioneered by Robert Mahony & co), but that will be the subject of another post :-)

Small blog post here: gtsam.org/2026/03/09/man…

Updated GTSam.org. Soon we’ll release an official 4.3. Watch this space :-)

even works on the phone :-)

Link is here: dellaert.github.io/26S-AMR/applet…

Applet is now live, and also does walking ;-)

Embedding fonts makes the svg in markdown look just as good as the latex path!

This might be too late for your IROS paper, but I vibe-coded a little DSL to easily generate tikz/SVG for factor graphs and Bayes nets :-)

SVG rendering is not super-faithful to tex, so I added a fgz2pdf command. Here's an example explaining the dynamics for a four-link mechanism.

Tried that now and it works after fixing a few bugs :-)

It's supposed to work even if you don't clone the repo (via npm install and npx), but I've not tried it. Let me know how that works :-).

@sirbayes :-)