Sabitlenmiş Tweet
cioran
95 posts


Am only half-way through this and the writing is already a masterpiece.
This is the polar opposite of AI slop.
Jaya Gupta@JayaGup10
English

I read the #book Principles of Neural Design recently.
It’s extremely dense, which you’d hate or love depending on how much you’re in awe with how our brain works. I totally loved it! The book is unique in trying to explain the wondrous complexity of brain using few unifying principles, all of which can be traced to constraints evolution faces, especially with energy efficiency.
The central insight from the book is this:
👉 Brains maximize information (bits) per ATP
Consuming energy and producing ATP is hard. The organism has to work to get energy. During evolution, inefficient designs get outcompeted by efficient designs so we should expect to see efficient designs. For brain, this means squeezing max information and computing using least amount of energy.
From this insight, we could derive the following principles:
1. Send only what’s surprising. Why waste energy in sending what can be predicted/estimated efficiently. This explains why predictive coding makes sense for the brain.
2. Minimize wire / compute locally. Since wiring is expensive and sending signals across length requires energy, brain does as much computation locally as possible (and sends only sparse computed results/commands at greater lengths). This explains why brain has “regions” (such as for language) since local computation minimizes wire.
3. Send information at the lowest acceptable spike rate. Because faster spike rate requires thicker axons and more ATP, brain sends sparse, low-mean firing rate signals. The lower bound to how slow signals you can send is determined by requirements for fast reaction (this is why we do get few fat axons from the eye into the brain, but most axons or then). It also explains how brain simply sends a simple sparse signal to local pattern generators near legs than then send precise signals to leg muscle for movement. Similarly, retina/eye does a lot of local computation and sends summaries only to the brain. There are two kinds of sparsity: lifetime sparsity (how often each neuron fires, across stimuli - averages to be 1Hz in V1) and population sparsity (how often each stimulus activates neurons - averages to be few percentage at a time in V1)
4. Firing codes match the distribution of natural data. Spike rates encode equal probability bins of natural data (often log scale), as that’s most optimal (maximizes Shannon entropy; it’s crazy how nature discovers mathematically elegant solutions way before humans did. Another example of it is how our inner-ear cells do Fourier decomposition of incoming waves)
5. Do analogue and chemical computation (wherever you can): much cheaper than digital spikes as they cost a lot more energy (but digital spikes can communicate far, so they have to be used). This means a lot of computation happens within dendrites. Some researchers believe because dendrites are so complex, each biological neuron is better approximated by a full multi-layer perceptron.
6. Make components irreducibly small: shrink to save materials/energy as much as physics allows (after a point, noise dominates, so you can’t shrink them lower)
7. Complicate to optimize: retina alone has 100 types of cells, specialist cells allow both efficient computation (because of their morphology) but also sparse, low-rate communication (when it spikes, downstream knows who sent the signal). A generalist neuron must use higher spike rate and downstream neurons must wait longer to integrate. This principle was most surprising to me because as an engineer, I’ve been feed simplify and generalize as principles throughout my life. But nature likes to complicate things.
8. Adapt, learn, forget: firing rate gets continuously re-adjusted (e.g. when you move to brighter room or a loud concert), without this, brain would be far more inefficient.

English
cioran retweetledi

@iishaparekh tell me you are vibe coding without actually saying so...
English

@beffjezos buddy, your feed is like rest of us..normie cuckoldry..are you building something or are you building something? ...come out of the closet and be a certified xhitposter already
English

@hardeep_gambhir the first order of fight club is you dont talk about...
English

thanks for proving your point by talking about it!
Crypto Tea@Cryptotea
We are living in a time where intelligent people have to stay silent to survive
English

dont install xitter
Stormi@Stormi_luv
You meet your 18 year old self, you’re allowed 3 words. What do you say?
English

Best NGO, i worked for ever!
Cormac@cormachayden_
our team ritual will be studied for generations
English



















