Adrien Taylor

Too busy/tired/lazy to find a convergence proof for your latest optimization algorithm? Let your computer do it! PEPit is a new Python package for computer-assisted worst-case analyses  (github.com/bgoujaud/PEPit or pypi.org/project/PEPit/). (1/5)

Very excited to be in Copenhagen for #EurIPS 🇪🇺 I am presenting an optimizer benchmark for diffusion model training (sunday @ PriGM workshop). it compares new methods (Muon, SOAP, ScheduleFree) to good old AdamW. Happy to chat anytime ❄️

@ErnestRyu @RaduIoanBot that's very cool, Ernest!

@RaduIoanBot This concludes my three-part Twitter series. Now that we have the mathematical results sufficient for a publication, I will post our work on arXiv on Monday. After a week or two of polishing the writing and gathering feedback, I’ll submit it for peer review. 14/N, N=14

I used ChatGPT to solve an open problem in convex optimization. *Part III* 1/N x.com/ErnestRyu/stat…

@EndgameL76909 @ErnestRyu Because you can cast the analysis problem as an optimization problem itself. If you want the details: francisbach.com/computer-aided…

@ErnestRyu @ErnestRyu can please explain why it's a 6-dimensional search problem. I tried searching for the answer but I couldn't find and it's bothering me a little

This is really exciting and impressive, and this stuff is in my area of mathematics research (convex optimization). I have a nuanced take. 🧵 (1/9)

@AIGuideOfficial Could you perhaps share the prompt, the full answers, and whether you actually checked the proof? :-)

I have resolved the open problems in the paper Open Problems and Two Riddles in Heavy-Ball Dynamics” (arXiv:2502.19916v1) using ChatGPT5. I have contributed to advancing the understanding of convex optimization. This result adds a new insight to the field, refining the conditions under which acceleration holds and potentially opening avenues for further research. While it may not constitute entirely “new mathematics” in the sense of a groundbreaking theorem, it does represent a significant step forward by settling a conjectured question with a rigorous proof, thereby enriching the mathematical landscape.

Learning rate schedulers used to be a big mistery. Now you can just take a guarantee for *convex non-smooth* problems (from arxiv.org/abs/2310.07831), and they give you *precisely* what you see in training large models. See this empirical study: arxiv.org/abs/2501.18965 1/3

The sudden loss drop when annealing the learning rate at the end of a WSD (warmup-stable-decay) schedule can be explained without relying on non-convexity or even smoothness, a new paper shows that it can be precisely predicted by theory in the convex, non-smooth setting! 1/2

@kfountou @inria_paris @Sierra_ML_Lab @ENS_ULM @CNRSinformatics @ERC_Research @Inria @CNRS @CNRS_Paris I never answered, sorry! Thanks Kimon :)

@inria_paris @TaylorAdrien @Sierra_ML_Lab @ENS_ULM @CNRSinformatics @ERC_Research @Inria @CNRS @CNRS_Paris Congratulations!

📰 #Article | Découvrez le projet CASPER, porté par @TaylorAdrien, chercheur au sein de l'équipe-projet commune @Sierra_ML_Lab (@inria_paris /@ENS_ULM/@CNRSinformatics), qui vient d’obtenir une bourse #ERCStG. Félicitations Adrien 🙌 ! 👉 inria.fr/fr/vers-des-al… [🧵...]

My book is (at last) out, just in time for Christmas! A blog post to celebrate and present it: francisbach.com/my-book-is-out/

Recruiting Post-Doctoral students in Machine-Learning, Optimization or Regulation in AI at TSE or Paris. Write to jerome.bolte@tse-fr.eu

We are looking for a post-doc in machine learning for astrophysics/astronomy ! pepr-origins.fr/2024/10/09/pos…

📰 #Article | Découvrez le projet ARTIFACT, porté par Justin Carpentier, responsable de l’équipe-projet commune WILLOW (@inria_paris @CNRSinformatics @ENS_ULM), qui vient d’obtenir une bourse #ERCStG. Félicitations Justin 🙌 ! 👉 inria.fr/fr/bientot-des… [🧵...]

@peter_richtarik @inria_paris @Sierra_ML_Lab @ENS_ULM @CNRSinformatics @ERC_Research @Inria @CNRS @CNRS_Paris Thanks Peter!

@inria_paris @TaylorAdrien @Sierra_ML_Lab @ENS_ULM @CNRSinformatics @ERC_Research @Inria @CNRS @CNRS_Paris Congrats, @TaylorAdrien !!

@FranckIutzeler @inria_paris @Sierra_ML_Lab @ENS_ULM @CNRSinformatics @ERC_Research @Inria @CNRS @CNRS_Paris Merci Franck :)! A bientôt!

@inria_paris @TaylorAdrien @Sierra_ML_Lab @ENS_ULM @CNRSinformatics @ERC_Research @Inria @CNRS @CNRS_Paris Bravo @TaylorAdrien !

@bremen79 @NicLoizou @gautamcgoel @MatharyCharles @baptistegoujaud same thing here; I guess it is a bug!

@TaylorAdrien @NicLoizou @gautamcgoel @MatharyCharles @baptistegoujaud Thanks to both, these two papers are indeed very useful! (Is there any other reply to your post? It says 5 replies, but it does not show any to me.)

Optimization people, how do you call this property? There exists L>0 such that f(y) - f(x*) <= nabla f(y)' (y-x*) - 1/(2 L) ||nabla f(y)||^2 where x* = argmin_x f(x) This is clearly satisfied by a convex L-smooth function, but it is weaker.

@NicLoizou @bremen79 @gautamcgoel @MatharyCharles With also convexity, I think it is indeed QG+ from Nicolas’ reference! (See arxiv.org/pdf/2205.15033 Def 1.1, and Thm 2.6), so « smoothness » wrt the optimal set + convexity. My current understanding tells me that convexity not only with x* is needed here @baptistegoujaud ?

@bremen79 @gautamcgoel @MatharyCharles For this, check out the upper conditions in the paper: proceedings.mlr.press/v130/guille-es… I believe your exact condition might be related to one or two conditions there. Remark: Your condition could also be related to the Polyak steps! ;)

@b_stellato @EPrinceton Congrats Bartolomeo!

Honored to receive the Howard B. Wentz Jr. Junior Faculty Award buff.ly/3R48BCb! Many thanks for my amazing colleagues at @EPrinceton for nominating me!

@adfillon … reviewer est un travail non rémunéré, très peu reconnu, et extrêmement chronophage dans un monde où le temps manque. Il semble raisonnable qu’une review n’identifie que quelques fautes importantes, c’est suffisant: pas besoin de les trouver toutes pour juger la qualité (2/2)

@adfillon Bonjour! Merci pour ce thread; en tant qu’auteur et reviewer, je suis très d’accord avec un certain nombre de points (je mettrais, perso, les conflits d’intérêts et les biais des reviewers en premier dans la liste). Par contre, … (1/2)

Le processus de la revue par les pairs est cassé. Un (long) fil🧵⬇️⬇️

@AcadSciences @Inria_Saclay @ENS_ParisSaclay @3DSfrance @inria_sophia @univbordeaux 🔴Félicitations à @julienmairal, lauréat du Prix @Inria-@AcadSciences Jeunes chercheurs🏆👏 Chercheur @inria_grenoble-@ugrenoblealpes, il explore les limites de l’apprentissage machine et de la vision par ordinateur dans de multiples domaines : inria.fr/fr/julien-mair…