Ferdia

If you are coming to #ICLR2025 come say hello. We will be presenting on how to turn Neural Operators equivariant even when all you know is the lie algebra! Generally happy to chat all things #geometry and #optimization in deep learning! 🔗arxiv.org/pdf/2410.02698

I recently learned about a very cool fact: if you compose flows of two Hamiltonians, this is also Hamiltonian. What is more - you can write down the exact Hamiltonian!

@sp_monte_carlo Sounds like someone with great taste 😂

@sp_monte_carlo Good old days

In our preprint, for the 1st time we make neural operators equivariant with respect to PDE symmetry groups. These can be very complicated, and often only the Lie algebra is known, so a universal method is needed - a 🧵. 🔗 Read the full paper here: arxiv.org/abs/2410.02698

@sp_monte_carlo I think I meant Chapter 5... In any case I like the book

@sp_monte_carlo cambridge.org/us/academic/su… (especially Chapter 7) is nice for this

@docmilanfar @TachellaJulian I was a bit confused by the terminology too, but it seems that divergence-free is used here to mean divergence-free in expectation, not pointwise.

@TachellaJulian I don't see how the "divergence-free" denoiser in UNSURE is actually divergence-free. once you have an estimate of η, f(y) = y + η_hat ∇ log p(y) can obviously be written as the gradient of a scalar field, which is not divergence-free

📢New preprint 📢 UNSURE: Unknown Noise level Stein's Unbiased Risk Estimator arxiv.org/abs/2409.01985… Optimal self-supervised losses (SURE, R2R) require exact knowledge of the noise distribution, and alternatives like Noise2Void are suboptimal. There is middle ground🧵

@docmilanfar @EeroSimoncelli SURE(D) = ||D(y) - y||^2 + 2 sigma^2 div(D)(y) is not an expectation. The whole point is that it's an unbiased estimate of the test MSE, E[SURE(D)] = MSE(D)

These two remarkable formulas, derived for the case of additive Gaussian noise, can be generalized to many other of noise: cns.nyu.edu/~lcv/pubs/make…

@docmilanfar @EeroSimoncelli But that it just a consequence of the fact that E[SURE(D)] is convex in D

@docmilanfar @EeroSimoncelli The only thing missing in this proof is checking that it is actually the minimiser (i.e. not just a stationary point)

It's a full house at the Geometric Deep Learning workshop here in rainy/sunny Cambridge!

@asHauptmann @Subho1Mukherjee @caromitreka We give an overview of this theory and some theoretical results that have been proven for these methods, before focusing on the setting of PnP with linear denoisers and proving that convergent regularisation can be achieved through a novel spectral filtering approach.

@asHauptmann @Subho1Mukherjee @caromitreka In the past decade, PnP has been shown to be a highly effective way of incorporating prior information (in the form of a denoiser) into solution methods for inverse problems in imaging. PnP methods remain relatively unexplored from the perspective of inverse problems theory.

I'm delighted to share that our paper (with @asHauptmann, @Subho1Mukherjee and @caromitreka ) on provably convergent regularisation using Plug-and-Play (PnP) methods has been published in FoCM! The paper is available, open access, here: lnkd.in/e9p9Sd2z

We are happy to host a joint seminar with the REMODEL programme, with Professor Takaharu Yaguchi speaking about Geometric Deep Energy-Based Models for Physics in our last CIA seminar!

We design non-expansive or averaged NNs and apply them to adversarial robustness, image denoising and Plug-and-Play methods. Imposing these constraints can be done while achieving high task performance, through a training approach that respects the underlying continuous system.

I'm glad to share that our paper (with @ElenaCelledoni, @MJEhrhardt, @dademurari, @b_owren, @caromitreka) on designing provably stable neural networks using connections to gradient flows in convex potentials has been published in Physica D! Check it out: sciencedirect.com/science/articl…!