Michael 英泉 Eisen@mbeisen
Too many threads on going on, so going to try to consolidate.
I don't think anyone objects to the core principle nominally at play here that if you put science out into the world, you are responsible for that work. This is what science is. I don't want to get distracted by questions of authorship or how responsibility is apportioned amongst authors - that's an orthogonal issue.
The expectation that you can trust the scientific outputs (and I'm intentionally broadening this beyond papers) of others is really a defining feature of science as a collective endeavor. And obviously, if a paper contains hallucinated references, fake citations, placeholder text, or obvious autogenerated junk, it’s hard to argue the authors exercised even minimal scholarly care.
People have tried to paint me and the others who have expressed concern about the new arXiv policy as somehow questioning this. We're not. To me something deeper shift is represented by that move, and I think it warrants at least acknowledgment - and IMO deeper discussion.
The value of preprint servers to the research community comes from them being fast, open, effectively unfiltered, and agnostic about correctness. A lot of great science is published first on arXiv and other preprint, and so is a lot of science that is poorly executed and often poorly presented. Since the existence of the later doesn't devalue the former, it's a bargain most people are happy with.
One of the things that kept this model afloat was the fact that producing a paper required some non-trivial effort, and therefore people inclined to produce works that could en masse disrupt the ecosystem could not actually produce them at scale.
AI has obviously shattered any remnant of connection between things that look like papers and scholarly output and effort (mind you, I think this is a good thing, but that's also a somewhat separate topic). **But the response to it has also broken something.**
arXiv (and other preprint servers) have always had to impose some kind of screening to keep out obviously inappropriate stuff, and I think most of us agree that asking "Is this an actual work of science?" before posting something is a reasonable thing for a preprint server to do (provided that the definition of what a work of science is is intentionally fairly broad).
However, the new policy is explicitly changing that bargain. The question is no longer "Is this a relevant scholarly work?" Rather it is becoming "Can we trust this authorial process?".
That is a HUGE shift.
Look, I understand why moderators feel existential pressure - the system isn't architected in infrastructure, processes or modes of use with a massive flood of AI-generated papers.
But there are some real risks in the new direction.
1) The thing that makes preprint servers different from (and better than) journals is that there is no gatekeeping. The new policy threatens this. Once moderation becomes about inferring authorial integrity, the boundary between “quality control” and “editorial policing” gets blurry. The fact that one of the 'punishments' is to force people to go through peer review before posting to arXiv (an idea too absurd to even mock), suggests that current leadership has a comfortable relationship to journal peer review that makes the risk that arXiv will become a journal in every meaningful sense more of a risk.
2) “Incontrovertible evidence” sounds, well, incontrovertible, but moderation systems take on a life of their own via various forms of procedure, precedent and social signaling. Today it’s hallucinated references. Tomorrow it could become stylistic mimicry. Slippery slope here.
3) The policy misdiagnoses the real problem. As I've said elsewhere, the issue is not “AI use” but the system that leads people to think it will benefit them to push slop onto arXiv. LLMs may amplify the negative effects of metric-driven academia, but they didn't create it.
To me we are at a fork in the road moment.
There is a world within our grasp where an alignment of preprinting and AI actually breaks the toxic stranglehold that traditional publishing has on science. A world where actual communication (not the facsimile of it we have today) takes place between people, between machines and from people to and from machines, around data and ideas in science.
But there is also a world where the preprint servers we love collapse in fear and a lack of imagination into irrelevancy and we lose to moment. I'm not saying this policy itself will cause that. But I am saying that it's not a good sign.
