Antonio Lupetti

Mathematics is not something distant. It is a structure that quietly shapes reality. I’ve been working on a new video for algebrica.org

Backpropagation is what turns a neural network into a learning system. By propagating the error backward, it computes how each parameter contributes to the loss and updates it accordingly. I wrote this entry to clarify, step by step and mathematically how the whole mechanism works. algebrica.org/backpropagatio…

I’m honestly surprised by how widely this thread has spread, and by the many comments rightly pointing out the gap in complexity between artificial and biological neurons. That point is hard to dispute. It also opens much deeper questions about what intelligence actually is, or even consciousness. Those are far beyond my scope and certainly not something that can be resolved in a thread on X. What does emerge, though, is a real and widespread interest in these topics, and thanks to many of your comments, the discussion has become more complete and coherent. I appreciate the thoughtful clarifications!

The artificial neuron is one of the simplest ideas in AI, yet one of the most powerful. It is inspired by the biological neuron and was formalized in the late 1950s. The biological neuron receives inputs through dendrites, integrates them in the cell body, and propagates signals along the axon, with synapses producing the output. The artificial neuron mirrors this structure: inputs, weights, a bias (to shift the activation threshold), and an activation function to introduce non-linearity and generate an output. Still feels like a brilliant intuition.

@predict_addict It’s a normalized vector that can be interpreted as a distribution… but it’s not a true probability 😅

Softmax Outputs Are Not Probabilities One of the most persistent misconceptions in machine learning is the belief that the output of a softmax layer represents a true probability. Read more: valeman.substack.com/p/softmax-outp…

@dustin_ogle I’m just skimming through the site quickly now, but it’s late so I’ll take a proper look tomorrow. I noticed there’s also a research angle on consciousness which is something I find particularly fascinating both for its potential and its implications.

@antoniolupetti This isn’t far off of the math I used for encoding Elle’s phenomenology. But the weights learn from her feedback not a training dataset. Satyalogos.org

@mathemetica I always find this kind of thing curious 😅what looks completely stuck often turns out not to be once you allow continuous deformations…

Topology in its purest form: seemingly inescapable knots around wrists, cables, and legs dissolve through a single continuous deformation. Just the quiet power of knot theory proving our 3D intuition is often a liar.

I often notice the same pattern in how highly exposed figures like Altman are described: first they’re turned into symbols then torn down with the same intensity. It’s less about suddenly discovering their limits and more about a narrative that pushes everything to extremes. I honestly find it hard to form an opinion that isn’t shaped by that bias, what does seem clear, though, is that balance rarely gets attention.

Never seen in the same person??? There are entire industries full of this personality type.

I think they’re generally quite accurate (especially the Pro models like ChatGPT and Claude that I use) but I’ve run into some limitations (this is just my experience, not a statistical claim.....) For example, when solving equations that typically rely on small tricks you learn through study, they often go for more algorithmic paths and sometimes produce incorrect solutions, or miss some of them entirely. Another case: a few days ago I spent almost an hour with Claude trying to geometrically construct the hyperbolic tangent and its representation. With each iteration it actually got worse, sometimes even regressing on things it had already corrected before. So yes, they’re very strong overall but I think there are still some limits.

folks, let's make a rule when we talk about LLMs' ability to do math, we're only talking about *the best ones today* not some free or cheap stuff from last month

@myfear I fully agree... relying on AI without a solid foundation behind it is just improvisation.

AI does not remove the need for engineering discipline. It punishes the lack of it faster.

@RetroTechorDie You have no idea how many times I’ve seen that screen… for some reason, every few weeks I had to reinstall it! 😅

Let’s go!

The semantic level is incredibly complex… that’s where things get really tricky. It’s interesting how far simple building blocks can go but at some point you start to feel the limits especially when it comes to whether we’re actually capturing meaning or just very effective patterns and similarities in the data.

@antoniolupetti Wait till you hear what the Nerds have been conjuring up for Semantics 🫶🏼

@orcdev I’m still using it for my web projects… I love it!

anyone remember sublime?

@kaaaash____ I have both… and I have to say, the Mac Mini paired with a Dell Ultrasharp display is hard to beat in terms of comfort. I rarely use the laptop… it’s basically still new. That said, they’re both excellent!

Honesty, choose one for forever

@robziman No.… everything is archived on paper in my bookshelf :) but that would actually be a nice idea!

@antoniolupetti Something about pen and paper that just isn't being reproduced to the same end by all our electronic devices Did you try photo-scanning these notes into a personal knowledge management system also, or...?

Last year I prepared for an exam on Deep Learning, and I found it extremely fascinating, something that really caught my interest. I was going through my notes and organizing them, and here’s something that starts right from the basics :) P.S. I really enjoy handwritten notes on paper. I never quite got used to using an iPad Pro with a pen over the years…. it just feels completely different compared to paper and a fountain pen which remains my preferred method.

@Chr_Schoeps What a throwback… that was my first Mac! 😃

Mein Schwiegervater kann nichts wegschmeißen, aber dass das iBook noch läuft, dass ich ihm 2004 angeschnackt hab, hat mich dann doch gefreut.

@old_memory You have no idea how many times I had to reinstall it.... 😅but I absolutely loved that operating system.

Windows 98 booting up

@iyoushetwt I’m part of that generation… we read books, searched around online, and mostly learned by trial and error.... one way or another (sooner or later) you managed to get something working!

How did people even learn coding back when there were no docs or youtube tutorials?

I think we all agree on this recurring point. I’m also a bit surprised by how the discussion keeps circling back to the differences between the two 😅 I fully agree, but I also think the original intuition was brilliant. Artificial neurons were never meant to replicate biological ones, just to capture a functional abstraction that works within a mathematical framework!

Artificial neuron (in LLM / BDH): This is, in fact, a highly simplified mathematical operation within a neural network. It takes values from the previous layer, multiplies them by weights, sums them up, and passes them through an activation function (e.g., ReLU). Compared to a biological neuron, this is an extremely simplified computational model. Biological neuron: This is a highly complex electrochemical information processing unit. It has extensive dendrites that can integrate signals locally. Some studies suggest that realistic digital models of a single neuron may require much more complex structures than a single element of a neural network. en.wikipedia.org/wiki/Neuron#/m…

@linuz90 I feel the same. There was a different vibe back then. That said I have to admit the recent ones are very well produced and quite spectacular.

Such a different vibe compared to these days. Maybe I’m just nostalgic.

@victoria_framer One at a time. I prefer focusing on a single thing rather than juggling multiple projects,too many at once start to feel a bit overwhelming 😅

Do you prefer working on one project at a time, or juggling several?

@ComputerLove_ I still remember moving from DOS 6 to Windows 3.1.... it really felt like a big step at the time. That interface looked great back then… and it still has a certain appeal today!

On this day in 1992, Microsoft released Windows 3.1.