Melanie Mitchell

There's a quadrillion-dollar question at the heart of AI: Why are humans so much more sample efficient compared to LLM? There are three possible answers: 1. Architecture and hyperparameters (aka transformer vs whatever ‘algo’ cortical columns are implementing) 2. Learning rule (backprop vs whatever brain is doing) 3. Reward function @AdamMarblestone believes the answer is the reward function. ML likes to use pretty simple loss functions, like cross-entropy. These are easy to work with. But they might be too simple for sample-efficient learning. Adam thinks that, in humans, the large number of highly specialised cells in the ‘lizard brain’ might actually be encoding information for sophisticated loss functions, used for ‘training’ in the more sophisticated areas like the cortex and amygdala. Like: the human genome is barely 3 gigabytes (compare that to the TBs of parameters that encode frontier LLM weights). So how can it include all the information necessary to build highly intelligent learners? Well, if the key to sample-efficient learning resides in the loss function, even very complicated loss functions can still be expressed in a couple hundred lines of Python code.

🎙We’re back with a conversation between one of the greatest minds in AI research @MelMitchell1 of the @sfiscience and @ethanjb. Tune in for perspectives on AI capabilities, performance benchmarks, and the path forward that’s grounded in evidence: bit.ly/3PFyYAF

If you, as a CS Prof, are wondering whether you need PhD students at all now that you have wangled a subscription to Claude Code, your lab probably had a pretty depressing vibe to begin with--and 'em students are likely better off with you hanging out with Claude.. #AIAphorisms

x.com/i/article/2026…