Zach Nichols
693 posts

Zach Nichols
@CallmeZmoney
the detail of the pattern is movement



There's a quadrillion-dollar question at the heart of AI: Why are humans so much more sample efficient compared to LLM? There are three possible answers: 1. Architecture and hyperparameters (aka transformer vs whatever ‘algo’ cortical columns are implementing) 2. Learning rule (backprop vs whatever brain is doing) 3. Reward function @AdamMarblestone believes the answer is the reward function. ML likes to use pretty simple loss functions, like cross-entropy. These are easy to work with. But they might be too simple for sample-efficient learning. Adam thinks that, in humans, the large number of highly specialised cells in the ‘lizard brain’ might actually be encoding information for sophisticated loss functions, used for ‘training’ in the more sophisticated areas like the cortex and amygdala. Like: the human genome is barely 3 gigabytes (compare that to the TBs of parameters that encode frontier LLM weights). So how can it include all the information necessary to build highly intelligent learners? Well, if the key to sample-efficient learning resides in the loss function, even very complicated loss functions can still be expressed in a couple hundred lines of Python code.






Fort Bragg guy whose bloodstream is 90% methamphetamine exploded by a flying lawnmower dispatched by the philosopher-warrior class of a theocratic society….we are approaching the “real life Warhammer 40K” territory





Margaret Mead’s account of Samoan free love sparked the sexual revolution. Her husband, Gregory Bateson, ran OSS black propaganda & used anthropology as a CIA culture-cracking tool. He helped seed the '60s counterculture & influenced MKULTRA. Mead's ideas were turned into psywar.

















