Sabitlenmiş Tweet
📣📣 Preprint alert 📣📣
« A Lower Bound and a Near-Optimal Algorithm for Bilevel Empirical Risk Minimization »
w. @tomamoral, @vaiter & @PierreAblin
arxiv.org/abs/2302.08766
1/3
English
Mathieu Dagréou
378 posts
Mathieu Dagréou
@Mat_Dag
Ph.D. student in at @Inria_Saclay working on Optimization and Machine Learning @matdag.bsky.social
Paris, France Katılım Temmuz 2019
547 Takip Edilen505 Takipçiler
Mathieu Dagréou retweetledi
This is the way.
Jeremy Cohen@deepcohen
@jasondeanlee @SebastienBubeck @tomgoldsteincs @zicokolter @atalwalkar This is the third, last, and best paper from my PhD. By some metrics, an ML PhD student who writes just three conference papers is "unproductive." But I wouldn't have had it any other way 😉 !
English
Mathieu Dagréou retweetledi
Nesterov dropped a new paper last week on what functions can be optimized with gradient descent.
The idea is simple: we know GD can optimize both nonsmooth (bounded grads) and smooth (Lipschitz grads) functions, but smooth+nonsmooth satisfies neither property, so what can we do?
English
Mathieu Dagréou retweetledi
🚟 New blog post:
On "infinite" learning-rate schedules and how to construct them from one checkpoint to the next
fabian-sp.github.io/posts/2025/09/…
English
Mathieu Dagréou retweetledi
For evolving unknown PDEs, ML models are trained on next-state prediction. But do they actually learn the time dynamics: the "physics"?
Check out our poster (W-107) at #ICML2025 this Wed, Jul 16. Our "DISCO" model learns the physics while staying SOTA on next states prediction!
English
Mathieu Dagréou retweetledi
Back from MLSS Senegal 🇸🇳, where I had the honor of giving lectures on differentiable programming. Really grateful for all the amazing people I got to meet 🙏 My slides are here github.com/diffprog/slide…
English
Mathieu Dagréou retweetledi
❓ How long does SGD take to reach the global minimum on non-convex functions?
With @FranckIutzeler, J. Malick, P. Mertikopoulos, we tackle this fundamental question in our new ICML 2025 paper: "The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes"
English
Mathieu Dagréou retweetledi
I want to address one very common misconception about optimization. I often hear that (approximately) preconditioning with the Hessian diagonal is always a good thing. It's not. In fact, finding a good preconditioner is an open problem, which I think deserves more attention.
1/4
English
Mathieu Dagréou retweetledi
🧵 I'll be at CVPR next week presenting our FiRe work 🔥
TL;DR: We go beyond denoising models in PnP with more general restoration (e.g. deblurring) models!
A starting point observation is that images are not fixed-points of restoration models:
GIF
English
Mathieu Dagréou retweetledi
📣 New preprint 📣
**Differentiable Generalized Sliced Wasserstein Plans**
w/
L. Chapel
@rtavenar
We propose a Generalized Sliced Wasserstein method that provides an approximated transport plan and which admits a differentiable approximation.
arxiv.org/abs/2505.22049 1/5
English
Mathieu Dagréou retweetledi
It was received quite enthusiastically here so time to share it again!!!
Our #ICLR2025 blog post on Flow M atching was published yesterday : iclr-blogposts.github.io/2025/blog/cond…
My PhD student Anne Gagneux will present it tomorrow in ICLR, 👉poster session 4, 3 pm, #549 in Hall 3/2B 👈
English
Mathieu Dagréou retweetledi
Optimization algorithms come with many flavors depending on the structure of the problem. Smooth vs non-smooth, convex vs non-convex, stochastic vs deterministic, etc. en.wikipedia.org/wiki/Mathemati…
English
Mathieu Dagréou retweetledi
A really fun project to work on. Looking at these plots side-by-side still amazes me! How well can **convex optimization theory** match actual LLM runs?
My favorite points of our paper on the agreement for LR schedules in theory and practice: 1/n
Fabian Schaipp@FSchaipp
Learning rate schedules seem mysterious? Turns out that their behaviour can be described with a bound from *convex, nonsmooth* optimization. Short thread on our latest paper 🚇 arxiv.org/abs/2501.18965
English
Mathieu Dagréou retweetledi
Learning rate schedules seem mysterious?
Turns out that their behaviour can be described with a bound from *convex, nonsmooth* optimization.
Short thread on our latest paper 🚇
arxiv.org/abs/2501.18965
Aaron Defazio@aaron_defazio
The sudden loss drop when annealing the learning rate at the end of a WSD (warmup-stable-decay) schedule can be explained without relying on non-convexity or even smoothness, a new paper shows that it can be precisely predicted by theory in the convex, non-smooth setting! 1/2
English
Mathieu Dagréou retweetledi
Learning rate schedulers used to be a big mistery. Now you can just take a guarantee for *convex non-smooth* problems (from arxiv.org/abs/2310.07831), and they give you *precisely* what you see in training large models.
See this empirical study:
arxiv.org/abs/2501.18965
1/3
English
Mathieu Dagréou retweetledi
Our work on geometric disentangled representation learning has been accepted to ICLR 2025! 🎊See you in Singapore if you want to understand this gif better :)
Théo Uscidda@theo_uscidda
Curious about the potential of optimal transport (OT) in representation learning? Join @CuturiMarco's talk at the UniReps workshop today at 2:30 PM! Marco will notably discuss our latest paper on using OT to learn disentangled representations. Details below ⬇️
English
Mathieu Dagréou retweetledi
The Mathematics of Artificial Intelligence: In this introductory and highly subjective survey, aimed at a general mathematical audience, I showcase some key theoretical concepts underlying recent advancements in machine learning. arxiv.org/abs/2501.10465
English
Mathieu Dagréou retweetledi
My book is (at last) out, just in time for Christmas!
A blog post to celebrate and present it: francisbach.com/my-book-is-out/
English
Mathieu Dagréou retweetledi
When optimization problems have multiple minima, algorithms favor specific solutions due to their implicit bias. For ordinary least squares (OLS), gradient descent inherently converges to the minimal norm solution among all possible solutions. fa.bianp.net/blog/2022/impl…
English
Mathieu Dagréou retweetledi
🍏🍏🍏 Come work with us at Apple Machine Learning Research! 🍏🍏🍏
Our team focuses on curiosity-based, open research.
We work on several topics, including LLMs, optimization, optimal transport, uncertainty quantification, and generative modeling.
Infos 👇
English
Mathieu Dagréou retweetledi
🏆 #Distinction | Toutes nos félicitations à @gerardbiau (Centre @Inria @Sorbonne_Univ_), directeur de #SCAI et spécialiste des dynamiques statistiques dans les algorithmes d'IA, qui a été élu à l’@AcadSciences 👏.
sorbonne-universite.fr/presse/gerard-…
Français