Zebang Shen

"First Proof" (by Mohammed Abouzaid, Andrew J. Blumberg, Martin Hairer, Joe Kileel, Tamara G. Kolda, Paul D. Nelson, Daniel Spielman, Nikhil Srivastava, Rachel Ward, Shmuel Weinberger, Lauren Williams): arxiv.org/abs/2602.05192

EPFL's "Optimization for Data Science" PDF: n.ethz.ch/~jiaxie/gradua…

Your brain doesn't forget - it just loses the keys to unlock memories This paper introduces key-value memory architecture that separates storage and retrieval representations in brain memory systems, optimizing for both fidelity and discriminability. ----- 🧠 Original Problem: → Traditional memory models rely on similarity-based retrieval, limiting their ability to optimize separately for storage and retrieval → Current models can't explain how memories persist for decades despite rare access or how forgotten memories can be recovered ----- 🔑 Solution in this Paper: → The paper proposes a key-value memory system where inputs are transformed into two distinct representations: keys for memory addresses and values for memory content → Keys optimize for discriminability in retrieval while values optimize for storage fidelity → The hippocampus acts as key storage, while neocortex serves as value storage → Memories are accessed by matching queries to keys, then retrieving values weighted by match strength ----- 💡 Key Insights: → Memory failures occur due to retrieval issues, not storage limitations → Information once stored is never permanently lost → The brain implements error correction through attractor dynamics → Hippocampal representations optimize for discrimination while neocortical ones optimize for semantic content ----- 📊 Results: → Model demonstrates recovery of "silent" memories without retraining → Achieves 99% accuracy on initial tasks and 95% on subsequent tasks → Shows graceful degradation instead of catastrophic forgetting → Outperforms flexible encoders trained to minimize reconstruction error

Using LaTeX is WAY easier now with AI. 🧵of my top tricks. 1) Tables. Screenshot a table from anywhere, paste into @overleaf and the AI generates the LaTeX code for the table. That just saved me 10 minutes of typing in code!

"Mathematical Essentials for Convex Optimization" PDF: www2.isye.gatech.edu/~nemirovs/CUPP…

According to dynamical systems, psychiatric disorders can be seen as energy landscapes with peaks and valleys. OCD has deep valleys where neural activity gets stuck, while schizophrenia has shallow valleys that let neural activity roam too freely, connecting unrelated ideas.

"Graph Theory and Additive Combinatorics" by Yufei Zhao (MIT) PDF: yufeizhao.com/gtacbook/gtacb… Video Lectures: youtube.com/playlist?list=… Book's website: yufeizhao.com/gtacbook/

Interpolating at uniformly sampled point can suffer from Runge oscillations. Using Chebyshev nodes resolves this issue. en.wikipedia.org/wiki/Chebyshev…

fluctuation

If you missed it: the Keras 3 release announcement is here ➡️ keras.io/keras_3/

- Shoot a photo with an iPhone through a pair of glasses with a certain prescription - Click on the screen until the "AF/AE lock" function appears. Use it - Remove the lense This is how the person who wears those glasses actually sees [📹 sakata_yoshi] twitter.com/sakata_yoshi/s…

David Attenborough is now narrating my life Here's a GPT-4-vision + @elevenlabs python script so you can star in your own Planet Earth:

Forget all the police suspect chase scenes you know.

As a droplet touches a liquid surface it bounces gently, kept afloat momentarily by a thin air cushion. As the air dissipates the droplet comes in contact with the liquid. The droplet undergoes a multi-step coalescence journey: with each interaction, it merges and reduces in size, transitioning into a smaller droplet, until it eventually fully merges with the liquid below.

Chaos in the 5 body problem: five point masses under their mutual gravitational interactions. After a bit of a dance, they all depart into the void. 🎃

@Rainmaker1973 flocking

What happens when you throw an apple from an offshore oil rig [📹 Michael Young Brown] twitter.com/hurtingsoulsig…

Chaos and the double pendulum: the configuration space of the double pendulum is a torus (a circle for each angle coordinate) so we can plot the evolution of such a pendulum in two ways: either as a "real pendulum" in physical space, or as a point in configuration space :)

«There’s always another chance»

OpenAI: AI will kill us. Anthropic: AI will kill us. InflectionAI: AI will kill us. Nvidia:

Anisotropic diffusion is fun! It can also cause SGD in deep learning to get attracted to *higher* training error saddle points that nevertheless have *lower* test error than local minima with lower training error. Details here: arxiv.org/abs/2306.04251