Alexander Bates

So, some people are asking me why this EON fly video doesn’t show real ‘uploading’ since it does simulate a real connectome. The most important reason is that the functional parameters that define the dynamic behavior of individual neuron and synapse types in the connectome are unknown. Instead, they used an existing model (nature.com/articles/s4158…) which substitutes these with guessed parameters and grossly simplified dynamics. As made clear in that older paper, these are not sufficient to recreate the activity patterns that would be seen in the real fly. The simplified dynamics would not, for example, be able to choreograph the timing of leg muscles during walking or grooming, or the dynamics of the compass neurons encoding the fly’s heading direction, or the myriad other neuronal dynamics that make up the fly ‘mind’. So not an ‘upload’ by any reasonable definition. In fact, the simplified dynamics they used have only been demonstrated to approximate gross correlations along major sensory-motor pathways for a handful of neurons. For example: activating a sugar sensing neuron causes gross downstream activation that elevates the activity of feeding neurons. It is this handful of very, very crude and basic correlations in the simulated connectome that are being used to drive the EON simulated fly. If they had said that from the start, then I would have had no issue. But instead, they made the bold claim that they had “uploaded a fly” and presented a video of said fly walking over a landscape with highly articulate legs, visually navigating through the terrain to a food source, grooming its antenna with eerily fly-like leg motions, etc. Any reasonable layperson would assume that these visually exciting articulations are the ones being controlled by the simulated brain’s dynamics instead of being faked by computational add-on routines. There are now many secondary reports of this on YouTube and all of them seem to make this reasonable assumption (e.g. youtube.com/shorts/Z7NNP1Z…). And who could blame them? Many neuroscientists also made that assumption before EON started to spell out what was really behind the video millions of views and over a day later. To make clearer just how misleading EON Systems’ video is and how outlandishly laughable their ‘uploading’ claim is, below is an imagined back-and-forth discussion between a [Reasonable Layperson] and a [Neuroscientist] trying to explain to them what is really behind the video: [Reasonable Layperson] “Look at the complicated leg motions as the fly walks… the timing of all those dozens of individual muscles being controlled by the dynamics of the simulated neurons… and they say that they used no reinforcement learning to tune parameters, just the connectome… that is really impressive!” [Neuroscientist] “Well actually no… those leg movements are actually coming from a program unrelated to the connectome. The connectome used didn’t even include the central pattern generator circuits in the ventral nerve cord responsible for controlling leg muscles.” [Reasonable Layperson] “Oh… so in what sense is the simulated connectome controlling walking?” [Neuroscientist] “It looks like they just found a few neurons in the brain connectome that are correlated with right/left/forward motion and used these to ‘steer’ the pretend walking routine.” [Reasonable Layperson] “Oh… But the activations of those ‘steering’ neurons are reflecting the complicated dynamics of tens of thousands of simulated neurons in the fly visual system as it moves through the virtual world, avoiding objects and heading toward its visual goal, right?” [Neuroscientist] “Well actually no … The visual system and virtual world are essentially ‘decoration’… the flashing dynamic neural responses as the fly moves through the virtual environment are designed to give the viewer the impression that the simulated fly is actually seeing the world and making walking decisions based on those visual responses. But, in fact, they could turn off the lights and the fly would behave identically.” [Reasonable Layperson] “Oh… so how does the fly walk toward the food then?” [Neuroscientist] “Well… it looks like they simply imposed an odor gradient in the virtual environment that is centered on the virtual food. The fly has two sets of odor receptors (right and left) that sense this gradient and the activation of these in the connectome is correlated with the activation of the ‘steering’ neurons. So if the left odor neuron activates more than the right then the fly steers left.” [Reasonable Layperson] “Oh… so it is like one of those toy cars that moves toward a light because it has right and left light sensors cross-connected to right and left motors… Gee, I thought a fly was more complicated than that.” [Neuroscientist] “Well actually a real fly is. Real flies have dozens of behavioral states that allow intelligent behavior in a complicated visual and sensory environment. In fact, a real fly contains a set of neurons which act as an internal compass updated by the visual environment and the fly’s walking.” [Reasonable Layperson] “Oh… and their connectome has those internal compass neurons?” [Neuroscientist] “Yes. They used the full brain connectome that contains those compass neurons.” [Reasonable Layperson] “...And their compass neuron activations are tracking the visual environment just like in the real fly?” [Neuroscientist] “Oh sweet summer child… those compass neurons exist in their connectome simulation, but no one knows enough about their functional parameters (synaptic weights, time constants, etc.) to simulate them accurately. They light up in pretty patterns totally unrelated to how they would in a real fly walking through that visual world.” [Reasonable Layperson] “Oh… and the complicated leg movements it shows during antenna grooming… is that also just a faked recording?” [Neuroscientist] “Yes. All the complicated leg motions shown during grooming are faked by a hard-coded program. But they turn that fake routine on or off by looking at some neurons in the connectome that are correlated with actual grooming behavior triggered by dust accumulation on the antenna… well really they fake the dust too by just activating a set of neurons after a delay.” [Reasonable Layperson] “And what did EON Systems do? Did they acquire the connectome? Did they determine the neurotransmitter types? Did they do the calcium imaging experiments to determine the steering and grooming neurons? Did they make the mechanical fly model?” [Neuroscientist] “No. Those were all done by real labs who were kind enough to carefully write up their results in open journals and to post their results and code openly online…. It looks like Eon Systems just took their code and put it together with a virtual environment designed specifically to trick viewers by triggering behaviors in misleading ways.”

This is not a fly uploaded to a computer theverge.com/ai-artificial-…

So Eon put out a more detailed blog post, my takeaways: Vision inputs are based on (Lappalainen, 2024) (connectome-constrained model trained on an optic flow task) and are "somewhat 'decorative'" and "do not currently substantially influence our behavioral outputs" Walking uses "slight modifications to existing NeuroMechFly controllers, trained to imitate the walking behavior of the fly" so, imitation learning, not connectome-driven Brain-body mappings "can be somewhat arbitrarily chosen by hand (as is our case)" "Best understood as a research platform and a demonstration platform" "Should not yet be interpreted as a proof that structure alone is sufficient" I think this is cool work ! bringing together several existing projects in a sensible way, but I have to say the announcement was premature and insufficiently detailed Their detailed write-up: eon.systems/updates/embodi…

We've uploaded a fruit fly. We took the @FlyWireNews connectome of the fruit fly brain, applied a simple neuron model (@Philip_Shiu Nature 2024) and used it to control a MuJoCo physics-simulated body, closing the loop from neural activation to action. A few things I want to say about what this means and where we're going at @eonsys. 🧵

What's better than talking about simulating brains? Actually doing it. Apply to the San Juan Winter School on Connectomics and Brain Simulation sjcabs.com Deadline extended to Oct 24!