orientino

Looped models scale compute by iterating in depth, demonstrating great potential in reasoning. But when should the looping stop, and how to keep it trainable at tens of thousands of unrolled layers? FPRM solves both. (1/7)

Why does Adam work better than SGD? Does it really? Many isolated explanations exist but finally we believe the picture is clearer: there is no single consistent source, except for the batch size trend, jointly shaped by data and architecture. Basically, all of it matters? Yes.

#ICLR2026 Into mode connectivity, model merging, or permutation invariance? We show how optimization dynamics shape the loss landscape of merged weights. Come check it out! 📅 23/04 10:30AM – 13:00PM 📍 Pavilion 3 P3-1809 w/ @TheusResearch @DamienTeney @orvieto_antonio

@mihaibujanca @unireps @ELLISforEurope @orvieto_antonio @SaraKangaslahti this should work us06web.zoom.us/j/86280859643

@unireps @ELLISforEurope @orvieto_antonio @SaraKangaslahti Both QR codes and the zoom link on the ELLIS website seem to be broken

📢The next UniReps x @ELLISforEurope speaker series event is happening on 26 February at 4:00 PM CET with @orvieto_antonio and @SaraKangaslahti . Don’t miss it!🚀

@VitalikButerin @DeanEigenmann this sounds straight out from the hitchhiker's guide to the galaxy lol

@DeanEigenmann Prediction: in 2035 regular printers will continue to suck, but 3D printers will improve, to the point that there will be at least a few cases of someone using a 3D printer to print out a flat document because it was the easiest tool they could make work for the task.

when I achieve billionaire status, I won't go to space. instead I will fix the main issue mankind faces today: printers

PSA: never, ever write "we use the same learning rate across all methods for fair comparison" I read this as "do not trust any of our conclusions" and then i move on. If learning rate tuning is not mentioned, it takes me a little more time to notice that, but i also move on.

@AdirRahamim1 nice! we worked on a related question: how does the optimization dynamics affect model merging? how do different optimization components interact with each other? arxiv.org/abs/2510.04686

Model merging is a game-changer for multitask learning without retraining. But why do some models merge perfectly while others crash? In our new paper we define and investigate Mergeability. #LLMs #MachineLearning #NLProc #ModelMerging

@zhuokaiz it would help to have a compiled .pdf for preview

Finally had time to prep intake materials for my eb-1a petition and realized I actually never had a proper academia-style CV 😅 Looked around, (surprisingly) didn't find many templates I liked, so I made one. Nothing too special, but I think it looks better, and has all the macros ready to use. Sharing it here in case it’s useful: github.com/zhuokaizhao/ac…

@_igorshilov Cool work! I have seen a similar idea before, where they also show the possibility of confining unwanted data to specific neurons in smaller networks: arxiv.org/abs/2307.09542

New Anthropic research! We study how to train models so that high-risk capabilities live in a small, separate set of parameters, allowing clean capability removal when needed – for example in CBRN or cybersecurity domains.

More on mlcontests.com/state-of-machi… e.g., - Python remains the most used language - Quantization (4-8bits) + LoRA is used for LLM - AutoML methods are becoming useful for narrow domains

A team with only one member has the highest win-rate

Interesting bits on the panorama of state-of-the-art ML competitions (e.g. Kaggle): (1/n)

@aaron_defazio Shameless plug that you may find interesting. We studied how the optimization dynamics affect the averaging of model weights arxiv.org/abs/2510.04686

It’s still common belief that the loss landscapes of neural networks are too non-convex for averaging of model weights to work. Meanwhile, the empirical evidence is overwhelming. It works! There is a whole tutorial on it this year. Adapt your beliefs or fall behind.

“To write well is to think clearly” - which is to say that by delegating more and more of your writing effort to AI, you are inviting the slow erosion of your own ability to think clearly.

Industrial Revolution brought environmental pollution, and Artificial Intelligence brings information contamination. It is getting increasingly serious. Pollution destroys the environment, but contaminated information destroys civilizations.

@giffmana I am using yabai and it disables the macos animations. The desktop transition is instant and comparable to i3. The only problem I have is that sometimes it is buggy, I need to reset it bc it stops autotiling

Does anyone around Zürich have a macbook with Yabai or Aerospace installed? Need new personal laptop, mbp has the best hardware, but i live and die with i3wm. I’d like to try them before buying one. Or any other idea how to try them? Want to see how instant desktop switching is

ppl in ML don’t realize how good they have it with arxiv, blogposts, and discord so many other fields still operate in a world of gatekept paywalled slop sites and year-long journal wait times legit feels like a third world country