Bread

Had not heard of context distillation when we wrote the paper back in 2024 but this is great stuff & way ahead of its time! Our initial paper showed us that prompts could in principle be converted into weight updates — and surprisingly fast with new advances like LoRA, and increasingly capable open source LLMs. We started Bread to take the next step: making LLMs learn at human speed, autonomously, from the real world, in the real world. That involves - Taking the same data from inference time - Using it to structure coherent weight updates repeatedly (like humans learning new concepts). - Doing that usably and reliably at scale (distillation to + from many model versions) Technically, that involves solving 1. Ingestion problem: how does incremental inference time data update the configuration of the bake? 2. Digestion problem: how do you structure updates to retain model coherence? 3. Learning infra problem: How do you build a fast training platform which depends upon a rapidly expanding repertoire of new models for on-policy updates? 4. Serving problem: how do you take advantage of GPU + bandwidth resources to serve inference & hot-swap new models created throughout inference time? The first research paper only put us at the starting line, and our work at Bread is running the marathon. Please check out the papers in this thread to see some early precursors to Baking! (wish we had known to cite them in the original paper). It’s satisfying validation to see others recognizing the value in converting prompts to weight updates, and I celebrate these prior works from Askell and Snell which bolster our commitment to and confidence in our mission.

Announcing Bread Technologies. We’re building machines that learn like humans. We raised a $5 million seed round led by Menlo Ventures and have been building in stealth for 10 months. Today, we rise 🍞