Sam Duffield

New open source: cuthbert 🐛 State space models with all the hotness: (temporally) parallelisable, JAX, Kalman, SMC

Blog Post - Optimizers and ODEs A continuous-time view of gradient-based optimization: starting from the observation that integrator choice matters in physics simulation, and transferring that insight to understand modern optimizers.

@cthorrez Oh this totally sounds like a challenge 🤩 would love to see if we can beat it with cuthbert 🐛 Input information = timestamps + {win/lose/draw} ? Performance metric = result prediction accuracy/NLL ?

even though it's old as hell it hasn't been beaten for general-purpose use by any other system which uses the same input information

Glicko 2 is still the GOAT for online rating systems

@octonion General bivariate Poisson family with some free parameters which you then fit to data doi.org/10.1111/1467-9…

For a baseline soccer model in a top club league, which model do you prefer? Poisson Dixon-Coles Bivariate Poisson with covariance Diagonal-inflated Poisson Conway-Maxwell Other?

Struggling with minibatch noise in Stochastic Gradient Bayesian Inference? Want your chains to naturally run on stochastic hardware? Introducing — Stochastic Gradient Lattice Random Walk! New work from @NormalComputing in collab with @zierhjmensch, @adnarim066, @wellingmax, and the dream team at normal computing, @Sam_Duffield, @MaxAifer and @ColesThermoAI.

It was my birthday this week, it really had to be done

Excellent to see! Unmissable for anybody interested in learning and inference for dynamical systems, at scale and in JAX.

Repo: github.com/state-space-mo…

Come check it out if you're interested in time series, Monte Carlo, sequential problems. We've got a suite of fun examples, lots more to add - contributions welcomed! Super fun work with @sahel_iqbal and @AdrienCorenflos 🙌

New open source: cuthbert 🐛 State space models with all the hotness: (temporally) parallelisable, JAX, Kalman, SMC

We just released cuthbert, a JAX library for state-space model inference. It provides various algorithms for Bayesian filtering and smoothing in one clean, functional API. Repo: github.com/state-space-mo…

@andrew_n_carr There are actually infinitely many different ODEs (and SDEs) that do the job arxiv.org/abs/2601.07834

in diffusion models, we typically think of the reverse process as a random walk that slowly builds an image from noise. This is described by an SDE: d𝐱 = [ 𝐟(𝐱, t) - g(t)² ∇ log pₜ(𝐱) ] dt + g(t) d𝐰 (notice the full g(t)² weight on the score function and the random noise term d𝐰.) but it's kinda cool that you can silence the noise entirely. i.e., there exists a deterministic ODE that generates the exact same probability distributions at every moment in time d𝐱 = [ 𝐟(𝐱, t) - ½ g(t)² ∇ log pₜ(𝐱) ] dt this is cool because the only difference in the steering (drift) between the random path and the smooth path is exactly halving the score term (½ g² vs g²). this precise adjustment perfectly compensates for the lack of random diffusion. AND if you simulated a billion particles using the random SDE and a billion using the smooth ODE, the resulting cloud of points would look identical at every time step t, even though the individual paths are completely different. because the ODE is deterministic, you can run it backwards! you can take a real photo, run the ODE in reverse to find its exact "noise code," and then run it forward to recover the original image perfectly. this is impossible with the SDE because the random noise scrambles the specific path. thank you mr Fokker-Planck equation

Very nice little preprint from @Sam_Duffield on the general solution to the inverse problem for the Fokker-Planck equation. I particularly enjoyed the use of Liouville's theorem!

Paper for full details. The proofs draw on ideas from vector calculus and Fourier analysis which was really fun to work through arxiv.org/abs/2601.07834

Here is the decomposition: I show that the scalar ϕ is unique(!) but you can choose and Q or D. In diffusion the ϕ terms represent the "probability flow ODE" but there are actually many ODEs which satisfy p(x,t) depending on your choice of Q

New preprint! A Complete Decomposition of Stochastic Differential Equations I characterise all possible SDEs that satisfy given time-dependent marginals p(x,t)

Grateful to have been part of the final stretch of CN101 bring-up and help validate Thermodynamic Generative AI in silicon. Massive kudos to the Normal team for moving this fast. Such an incredible group of passionate, brilliant people. 2026 is going to be a fun one :)

Usual MCMC algorithms are typically guaranteed to work well when sampling from target distributions for which i) mass concentrates well in the centre of the state space, and ii) the log-density is smooth and of moderate growth. Outside of this setting, things can go poorly.

@sp_monte_carlo @jasondeanlee I actually really like the oracle/tensor/dynamical distinction though

@sp_monte_carlo @jasondeanlee Tensor order 1 + L/d

Exactly. Many of the matrix preconditioning methods, people call 2nd order, but really are much closer to first order. Eg muon, shampoo, etc