Patrick Mineault

Excited to release what we’ve been working on at Amaranth Foundation, our latest whitepaper, NeuroAI for AI safety! A detailed, ambitious roadmap for how neuroscience research can help build safer AI systems while accelerating both virtual neuroscience and neurotech. 1/N

NEO’s Hands An API to the Physical World

There's a surprising amount of neurosymbolic work at the edge of capabilities in robotics. There's a case to be made that there are > 0 odds that (embodied) AGI won't look like undifferentiated end-to-end trained networks.

@neurosp1ke Sick! I'm going to try this with an so-101 to see what it can do with a crappy arm

At the beginning of this auto-calibration video can see the cell overview with Stereolabs ZED Mini on 3DoF mini Cambot mounted above the head/context camera of the TRLC DK-1 which is a dual arm 2x 6DoF QDD robot with wrist cams and parallel grippers (the leader arms are for teleop and not used for the agent experiment). Claude first calibrated intrinsics, extrinsics (handeye for wrist cams) and then switched to bundle adjustment including kinematic and adjusted joint offsets. The robot is mounted in a light box and there is a rubber mat on the table. The robot runs in impedance control mode which Claude further adjusted to let the robot actually reach target positions <1mm. It also measured disagreement between the two arms and the stereo cam by measuring the rubber mat plane by „touching“ it at multiple points.

Fable connected to robot rig is next level - it built all perception, motion and calibration primitives itself. Now learns to pick different objects… I was degraded to E-stop operator scene prep minion. Fable simply ignores half of my stupid questions or suggestions 😄.

🧠 What's the best way to spend compute & neural data to build brain-aligned models? Our new ICML 2026 paper charts the scaling laws across 8 neural datasets & 600+ vision models. 👇 🌐 multimodal-brain-scaling.epfl.ch w/ @yingtian_david & @martin_schrimpf

Richard Murray, a professor at Caltech, made this beautiful chart showing how the complexity of gene circuits (as measured by their number of "parts," or components) has scaled over time. (I'm sharing it below with permission.) We can learn many things from this chart. First, academic laboratories have been able to make some *really* complicated gene circuits. My friend, Jai Padmakumar, made the largest gene-circuit ever reported; it was described in a 2024 paper. Jai assembled 1.1 million bases of synthetic DNA into 110 distinct logic gates, and then partitioned that DNA across 66 strains of E. coli. Together, these engineered cells could compute the MD5 hashing algorithm. The problem is that the larger you make your gene circuit, the less "robust" or reliable the engineered cell becomes. Living organisms did not evolve to carry human-made gene circuits! Therefore, many synthetic biology efforts fail to scale into the real-world. The more complex a gene circuit, or the more genes it has, the less likely that it will be robust over time. More genes have more opportunities to break. (Note that this is not always the case in natural organisms. Many cells have evolved overlapping ways to regulate genes, such that if one breaks, others can fill in the gap. We're not good at emulating this synthetically, though.) The chart below shows this trend via the red, dotted line. Engineered cells that have been *commercialized* tend to have only a small number of engineered components; usually less than 10 synthetic genes in total. There is a drop-off in number of components as we move from the laboratory to the real-world. How can synthetic biologists solve this, and begin to build large gene circuits that are robust over time? Perhaps we should make it standard to grow engineered cells in a small bioreactor, perturb them with various stressors, and see how well the engineered cells hold up over time. We could record the number of generations that pass before a cell's functions break, and then report that value in the paper. (This is sometimes done, but not often.) Another option is to "merge" human-made designs, or AI-generated DNA, with continuous evolution. If we wanted to engineer a cell to break down plastic and recycle the atoms into a medicine, for example, then we could first build dozens of different gene circuit architectures (using high-throughput DNA assembly methods), put each gene circuit into a cell, and then do continuous evolution on each of them to see which one holds up best over time, with various stressors. We could sequence the populations over time, see which sequences hold up well, and use the data to train predictive models of "cellular robustness."

Today #opensource dexterity became accessible to everyone! 🤯 just follow along the step-by-step assembly video and build your own 17 DOF orcahand v2 from our $1,700 BOM kit

Interested in the principles shaping connectome architecture? Check out our latest in @CellCellPress led by @_francisnormand w/ @jchrispang, @TrangCao2016, J Cruddas, @m_gajwani, @Arshiya_San, @DrAlexHolmes, @StuartJOldham, & P Robinson. We derive a simple model using geometric eigenmodes that can capture cortical connectome topology and topography of mouse, marmoset, macaque, chimp, and human connectomes, as mapped using diffusion MRI and tract-tracing. Paper available here: sciencedirect.com/science/articl… Peep Francis' thread for more details!

We’re sharing the next major milestone in our non-invasive brain-to-text decoder research: Brain2Qwerty v2. Building on v1, which was published today in @Nature, Brain2Qwerty v2 is the highest-performing end-to-end pipeline capable of real-time sentence decoding from raw brain signals. It advances beyond character-level performance to decoding words and semantics, enabling accuracy for overall communication. We believe this research has the potential to make a real difference for the millions of people who suffer from brain lesions or disorders that prevent them from communicating. 🧵👇

It’s been a big week (month? year? decade?) for ultrasound and I’m getting dozens of DMs so I’m going to do another one of these “the science behind” threads. On this weeks’ edition: Aleph’s incredible images and the ultrasound science behind them. 🧵🔊 x.com/alephneuro/sta…

We can learn a model that provides shaped "process rewards" for robotic RL, that evolves automatically as the policy gets better. This improves performance on benchmarks, and works in the real world! Some fun new work with Raymond Tsao & @ajwagenmaker

Turns out you can invert the Bellman equation to recover an agent's world model from its value function. Excited by the potential applications of this work, lead by @_aletcher. My fave bit - RL agents implicitly model latent variables they were never trained to optimize for..🧵

神経科学×ロボティクスのおもしろ掘り出し物講義はっけん！ youtube.com/playlist?list=… 視覚とロボティクスの伝説的な研究者であるUC BerkeleyのJitendra Malik教授の 「Robots That Learn」全12回(第1回除く) がYouTube で無料公開されてます。 チラ見しただけでも、 ・赤ちゃんに歩き方を事前プログラムしないのと同じで、ロボットにも「前に進む」「エネルギー消費を最小化する」といった原則だけ与えると、最適な歩行が勝手に創発してくる。 ・物を回すといった複雑なタスクでは、ロボットの関節の角度だけでなく視覚や触覚を足すと成功率が跳ね上がる。 ・人間の遺伝子に組み込まれているのは報酬構造と学習のカリキュラムのみで、具体的な能力は環境との試行錯誤で身に付けていく。 のように、「生物の世界との関わり方を手本にして、ロボットに学ばせよう！」といった話が出てきて、非常に趣があります。 内容は以下の三部構成になってるみたいです。 ①生物のモーター制御の基礎（動物は実際どうやってるのか？） ②ロボットの運動スキル習得のパラダイム（どうやってロボットに教えるのか？） ➂ケーススタディ（歩行・ナビゲーション・操作etc） 興味ある方はぜひ！ （主はUC Berkeleyの回し者でもなんでもないただの日本人です。）

Can a robot acquire real-world dexterous manipulation skills from just human videos? Meet Video2Sim2Real: full-stack autonomous dexterous skill acquisition from a single RGB-D human manipulation video — without robot data or expert intervention. Project: video2sim2real.github.io Paper: arxiv.org/abs/2606.08828

So proud of @ChrisInterno who took @EeroSimoncelli and @olivierhenaff's straightening hypothesis to set a new SOTA across 3 AI/Physics video benchmarks (2 detection, 1 generation), beating the leading models by @AIatMeta @NVIDIAAI and others, also big thanks to our tiny but mighty GPU cluster @CSHL it's not everyday that we get to stick it to big tech 😉

They squeezed so much out of a neuron: Ca²⁺ imaging → organic-solvent clearing → 100-hour whole-brain 2P scan → tracing → rehydration → sectioning → expansion/FISH → multi-round RNA imaging → registration. doi.org/10.1016/j.cell…

My favorite part: ABC is a reproducible robotics stack. The website demos are not just cool videos: the underlying data, recipes, and infra are open, so people can actually inspect and build on them. Also excited about the >400h of sim teleop data coming soon. This should help the community iterate faster and study questions like sim-real co-training. One encouraging signal from our experiments: even with only ~2% simulated data mixed into co-training, we observed strong sim-real performance correlations. Excited to see what the community builds on top!

Working in robotics right now is what I imagine working with language models felt like in 2023. Everyone throwing things at the wall to see what sticks Pixel prediction (Cosmos), action prediction (VLA), reward prediction (TD-MPC), and representation prediction (JEPA). Different paths for the same problem The recipe that won in language was self-supervised pretraining at internet scale then light finetune on top. Only representation prediction runs that playbook. It learns from action-free video data so you can pretrain on YouTube and egocentric data then add a control layer. Everything else needs action-labeled data that doesn't scale As an RL maximalist, I used to hate LeCun's cake. Turns out he was right all along which is how I ended up a JEPA truther

(Continuing from previous post) 8/ Nejat et al (@HNyXJ): a council of 10 local LLMs scores 31 Predictive Processing studies against a 36-concept human-expert defined ontology (3 hypotheses derived from predictive coding), mapping a fragmented literature into a quantitative hypothesis space, with structured disagreement and a "temperature" metric. Local, auditable, reproducible. arxiv.org/abs/2606.05206 You can visualize the hypothesis space for local/global oddballs (and the shift between them) yourself here: hnxj.github.io/pages/gallery/ It's super cool (and fun!)

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵