Scott Erb
12.3K posts

Scott Erb
@sserb
Husband, Father, Coach, Leader, Innovator. The 1% Project. Improving 1% each day morally, mentally, and physically. https://t.co/inP1Z4vmun
This seems really bad and I don't know what to do about it: not so much the differences in political attitudes, that's fine, but there's a strong gender divide in belief on straightforward factual questions like "is nuclear energy low-carbon?" yougov.com/en-gb/articles…



AI Cannot Self Improve and Math behind PROVES IT! | Devsimsek So, I saw a LinkedIn post (forwarded by a friend, thanks again) that stopped my doom-scrolling dead in its tracks. The headline? “Researchers just mathematically proved AI cannot self-improve.” My first reaction was the classic developer response: “I called it earlier!” My second reaction was to actually read the paper. Turns out – yeah, we’re right. And the math behind is kind of uncomfortably elegant. The Dream They All Had The whole “AI singularity” narrative goes something like this: we build a smart AI, that AI improves itself, the improved version is smarter so it improves itself even faster, and then – boom – we either all live in utopia or become paperclips. This is called Recursive Self-Improvement (RSI), and it’s been the backbone of both AI doomer manifestos and Silicon Valley pitch decks for a decade. The implicit assumption is that an AI training on its own outputs would get better over time. Like compound interest, but for intelligence. Sounds reasonable, right? Yeah. About that. What the Paper Actually Says A recent arXiv paper – “On the Limits of Self-Improving in Large Language Models” – doesn’t just argue against RSI. It formally proves it’s self-defeating. The core idea: model the self-referential training loop as a dynamical system on the space of probability distributions. When a model trains on its own generated data (synthetic outputs), it’s not learning from reality anymore – it’s learning from a distorted reflection of itself. The paper proves that under a diminishing supply of fresh, authentic data, this system converges to a fixed point – a degenerate distribution with low diversity and high bias. The technical term is model collapse, and it’s been observed empirically too. But now there’s a formal proof that it’s inevitable, not just a bad luck outcome. In plain terms: the model doesn’t climb toward superintelligence. It slowly forgets what the real world looks like. # Oversimplified metaphor as code def self_improve(model, real_data_supply): while real_data_supply > 0: synthetic = model.generate() model.train(synthetic) real_data_supply *= 0.9 # diminishing fresh data return model # spoiler: this model is now dumber The proof also extends beyond single LLMs – it covers ecosystems of interacting models and multi-modal systems. So no, a committee of AIs feeding each other outputs doesn’t escape the problem. It might actually make it worse. The “Curse of Recursion” There’s a term I love from this paper: the curse of recursion. When your training data is increasingly polluted with your own synthetic outputs, the tails of your distribution disappear first. Rare but important patterns – edge cases, nuanced reasoning, outlier knowledge – get washed out. The model converges toward a bland, high-confidence, low-variance output space. You can see this empirically already. Ask a model that’s been RLHF’d into oblivion something unusual, and it’ll confidently give you a smooth, plausible-sounding, completely wrong answer. That’s collapse in slow motion. The math backing this is rooted in dynamical systems theory – specifically the idea that without an external “forcing function” (real, diverse, human-generated data), the system has no energy to maintain the complexity of the original distribution. It inevitably degenerates. What This Actually Means for the Industry This doesn’t mean AI stops improving. It means the self-improvement loop fantasy is dead – at least the version where you unplug the humans and let it run. What it does mean: - Human-generated data is irreplaceable. The “internet is running out of training data” problem just got mathematically formalized. You can’t fake your way out of it with synthetic data at scale. - RSI as a path to AGI is a dead end. At least the naive version – train → generate → retrain → repeat. It converges, but downward. - Curation matters more than quantity. A smaller dataset of high-quality, diverse, authentic human output beats a massive synthetic pile every time. Quality over quantity isn’t just a vibe – it’s thermodynamically correct. - We’re not getting a free intelligence explosion. The singularity crowd’s timeline assumptions might need some… recalibration. Personally, this makes me feel vindicated about something I’ve been quietly skeptical about: the idea that scale alone solves everything. It doesn’t. Data provenance matters. Signal quality matters. The universe doesn’t give you compound interest on noise. The Beautiful Irony Here’s what gets me: the very mechanism people proposed to transcend human limitations – training on AI-generated data to break free from the finite supply of human knowledge – is mathematically proven to destroy the model’s representation of reality. The escape route collapses into a trap. It’s like trying to bootstrap yourself off the ground by pulling your own shoelaces. The harder you pull, the more you reinforce failure. Does this mean AGI is impossible? (Even though I like to say yes, i neither have the enough research nor I want to comment on it) No. Does it mean the naive RSI path is a dead end? Mathematically, yes. The smarter path – and what labs are quietly shifting toward – is better data, better curation, better grounding in reality. Which, ironically, means humans stay in the loop longer than the singularitarians wanted. smsk.dev/2026/04/26/ai-…
@infantrydort As a 26 Army veteran and one of the 47% (and growing) of Americans who vote as Independents, I can objectively state that you are a traitor to the Constitution you swore an oath to defend, and to the Soldiers of all political persuasions you’re supposed to lead.
A Google researcher just proved AI consciousness is mathematically impossible. Not in 10 years. Not in 100. Ever. The argument is structural, not technical. Computation is a description of a process, not the process itself. For something to "compute," a conscious observer must first carve reality into symbols and assign meaning. Without that observer, there are only voltage gradients. The paper calls this the Abstraction Fallacy. The analogy that makes it click: > A GPU can simulate photosynthesis perfectly > It will never produce glucose > Simulation is not instantiation > Maps don't become territory The framework doesn't rule out artificial sentience entirely. It says if a machine were ever aware, it would be from its physical makeup, not its code. Scaling parameters cannot change category.
A drunk guy grabbed a guys wife ass in front of him... Is this the only way to respond?
. @MaxBoot makes solid points in today's @washingtonpost about the Navy Secretary's firing & the battleship program. The Iowa-class battleships were impressive — you should tour one. But they are museums for a reason. In the age of drone swarms, hypersonic missiles, and stealthy submarines, concentrating firepower in a handful of enormous, expensive, and easily targeted platforms is the wrong direction. The "Golden Fleet" sounds good. Distributed firepower across many smaller, faster, unmanned platforms is a better strategy. wapo.st/3P4vSWL
Interesting how it works Elon puts up his own money, rounds up the absolute best AI talent on the planet, leverages every connection he has to secure serious resources, and launches OpenAI in 2015 as a pure non-profit explicitly created to develop AI for the benefit of humanity, with zero profit motive and open research Then the “team” decides they want the bag They push Elon out, take control, and quietly flip the entire thing into a for-profit machine All while preaching the same sanctimonious lines on repeat: “We’re still mission-driven!” “AI for the good of humanity!” “We’d never abandon our principles!” The ultimate betrayal: Elon got zero equity. Not a single share. He funded it. He built the foundation. He got nothing while they turned his non-profit into their personal cash cow This is the level of betrayal and hypocrisy we’re dealing with And for the record.... this lawsuit doesn’t put a single penny in Elon’s pocket. Any win goes straight back to the non-profit to restore the exact mission he founded


