Aditya Cowsik

@michaelyli__ This really shows how much engineering you need to fully accept the bitter lesson!

Can a language model learn, end-to-end, what to keep in its own KV cache and what to throw away? Can it learn to forget while it learns to reason? Deep learning's central lesson: capability emerges from end-to-end optimization, not heuristics/strong inductive biases. But for efficiency, we rely heavily on hand-designed approaches. 🗑️ Introducing Neural Garbage Collection (NGC): we train a language model to jointly reason and manage its own KV cache, using reinforcement learning with outcome-based task reward alone. No SFT, no proxy objectives, no summarization in natural language. New paper with @jubayer_hamid, Emily Fox, and @noahdgoodman!

Check our new AI paper, The Persian Rug! arxiv.org/abs/2410.12101 We extract exactly the algorithm learned by the most well known model of neural network superposition, and distilled it into a set of weights resembling a Persian rug, which matches the learned loss exactly.