Patrick Mineault
5.4K posts

Patrick Mineault
@patrickmineault
NeuroAI researcher @ Amaranth Foundation, safety, open science. Previously engineer @ Google, Meta, Mila.


I worked on the fly connectome for over 6 years, and let me just say that y’all have to slow this hype train way down. Connectomes are amazing. Biomechanical models are amazing. Linking the two is awesome. But scientists at the HHMI Janelia Research Campus, Princeton, and other institutes have been working on this for years now, and it’s not clear to me what’s new in the below. And connectomes are still missing a LOT of information. We’ve had the connectome of the worm for over 30 years now, and we still can’t reliably simulate a virtual worm. For example, connectomes don’t capture information about neuromodulator or neuropeptide release sites or receptors. These molecules are constantly changing the properties of neurons in the brain in ways that we have yet to really understand. And we don’t yet understand animal behavior well enough to refine and/or evaluate whole-brain simulations effectively. @AdamMarblestone and @doristsao already made many of these points, as well as many other good ones, but I just wanted to also add my two cents.





Can language models learn useful priors without ever seeing language? We pre-pre-train transformers on neural cellular automata — fully synthetic, zero language. This improves language modeling by up to 6%, speeds up convergence by 40%, and strengthens downstream reasoning. Surprisingly, it even beats pre-pre-training on natural text! Blog: hanseungwook.github.io/blog/nca-pre-p… (1/n)







💼 Paid Opportunity: Join Neuromatch Academy and Climatematch Academy as a #virtual #TeachingAssistant this July. 8hrs/day, Monday to Friday during the course dates. ➡️ Learn more here: buff.ly/TMy0xF1 ➡️ Apply before 15 March buff.ly/DzszKRe



The Foundations of Tomorrow The transition to AGI needs to go well. We’ve deployed $350M+ to neuroAI, longevity, and more with the belief that the brain is the key to better, safer AGI. This is the first of many overview posts on our thinking. blog.amaranth.foundation/p/the-foundati…


How do cell types relate to function? Prodded by @AdamMarblestone's recent appearance on @dwarkeshpodcast, I break down the logic of Steve Byrnes' theory of the steering vs. learning subsystem, and answer why many cell types are better than few for instincts and primary rewards.





