cem retweetledi
In the era of continued pretraining and continued fine-tuning, loss of plasticity means leaving future gains on the table. We need a better theoretical understanding of loss of plasticity. See a great thread unpacking the dynamics. 👇
#ICLR2026 #ContinualLearning #DeepLearning
Amir Joudaki@AmirJoudaki
Neural nets don’t just forget. Sometimes, after long training, they lose the ability to learn at all. In our #ICLR2026 poster, we model Loss of Plasticity as gradient dynamics trapped in invariant manifolds: 🔴 frozen units, 🔵 cloned units. The video makes the traps visible.
English
