Casey Davis retweetledi
Godfather of neuroscience and author of The free energy principle (active inference), Karl Friston @KarlFristonNews , discussed with me his take on AGI, consciousness, why we will never fully understand the brain, and why humans tend to repeat their mistakes. Key moments:
- You'll know AGI has arrived when the system starts asking you questions out of genuine curiosity, not because it was prompted to
- You cannot hand an intelligent system a value function from the outside, it must learn its own, just as children do (in that sense, RL with assigned reward is the wrong direction)
- The only sustainable universal objective function is adaptive fitness: how well the agent fits and survives within its ecosystem
- Consciousness requires multiple layers: genuine agency, a self-reflective loop, and the ability to recognize your own states of mind
- True sentience may be impossible on standard computer architecture, because memory and processing are separate and cannot self-organize
- Understanding your own brain is philosophically impossible in the same way a ruler cannot measure itself
- Neuroscience is always "peeking behind" the Markov blanket indirectly: through imaging, electrophysiology, psychology — never seeing inside directly
- The only way to truly access the brain is to breach that boundary (e.g. neurosurgery), but a breached brain is no longer a normally functioning one
Watch the full interview and let me know what you think. Link below👇
English
