Han Xiao@hxiao
interesting position paper throwing cold water on autoresearch/ai scientist: LLMs can't jump.
The thought experiment is this: Take an LLM with a 1905 knowledge cutoff. Feed it every paper, every dataset, every equation of that era. Could it invent general relativity?
No.
Discovery isn't one thing. It's three. You can induce — generalize from data, which lands you at Newton plus some epicycles to explain Mercury's weird orbit. You can deduce — derive rigorously from axioms you already have, which never gives you new axioms. Or you can jump — invent the frame itself, decide that spacetime curves. That third move is the one that matters, and it's exactly the one induction and deduction can't reach.
Penrose put it as three worlds: Physical, Mental, Platonic. Data flows from the world into a mind fine. But the new law has to be discovered into the Platonic world first — and that step is the jump. LLMs are induction machines running over what already exists. Structurally, they don't take it.
I think it’s a warning to AI scientists/autoresearch against collapsing two very different things into one word.
Hill-climbing: LLMs are already superhuman here, and autoresearch in this sense is real and moving fast.
Abduction/leap/jump: a new frame that reorganizes the field, that is a different act entirely, and nothing about scaling induction suggests you get there.
Most of what Autoresearch ships today will be spectacular hill-climbing. The jump is still ours for now.
