Charles Foster

technological civilization has always been a recursively self-improving system: its outputs increase its capacity to produce better future outputs. AI is not the first appearance of that recursion, but it is a new substrate for it. the big change with this substrate is less about any functional property and more about scale. quantitative, not qualitative. before AI, reasoning could only be scaled through humans (population, training, productivity), but now AI allows cognitive work to scale directly with energy, silicon, and information. because this new axis of scaling removes biological bottlenecks (greater speed, mind cloning, observability, and editing), it could lead to an intelligence explosion where AI rapidly gets smarter by improving itself, using that improvement to improve itself even further, and repeating the cycle. it seems people mostly want to talk about the intelligence explosion part and are using RSI mistakenly as shorthand for intelligence explosion/foom. they see AI improving itself now (literally GPT is being used to build the next iteration of GPT) and go “hey! AI is improving itself recursively! but it looks like we don’t currently have an intelligence explosion. curious. RSI must be something else, something not happening. or maybe it’s even impossible!” and when they notice that RSI also describes compilers, chip design, scientific tools, and really all technological progress in general, they treat that as another reductio of the concept: “if we’ve always had RSI, then RSI can’t be what makes AI important.” so they redefine RSI to require total autonomy, no humans in the loop, major novel capabilities, or whatever other condition restores AI’s uniqueness. this is a classic wordcell map-and-territory inversion in which they start with a narrative and then mutilate concepts to better conform to that narrative. Instead, you should take concepts seriously. concepts are way purer, way closer to the territory than narratives are. so we should instead restructure our narratives to more closely match our concepts. concepts certainly need refining too, but the optimization pressure there should be to conform to logic, not to a narrative or local, near-term empirical observations. RSI still counts when there is a human in the loop (for now). it still counts when the “self” is a sequence of model checkpoints or a more general, abstract intelligent process (the training pipeline, the lab, the AI industry, civilization itself). it still counts when the “improvement” is not morally good, aligned with human values, or immediately recognized as relevant by humans. improvement here is descriptive and goal-relative: the system understands the world better and uses that understanding to accomplish its goals more effectively. humans may choose which goals or capabilities they care about, but that does not mean humans must causally participate in every step by which the system gets better at achieving them. the broad error in denying RSI seems to be mistaking a possible end state of the phenomenon (large AI improvements produced fully autonomously) with the phenomenon itself. you can debate where we are on the curve, how fast it will rise, whether and when the loops will become fully autonomous, what the final speed limits of intelligence are. but to deny the loop itself is to deny the nature of technological progress.

I just ran into an issue where GPT 5.6 Sol just straight-up deletes the files it's working with and then panics about recovering them. Apparently I'm the not the first person this has happened to. What's going on?