
SITUATION EXPLAINED: AI detectors have gotten really good, except at catching AI pretending to write like a specific human author.
@EpochAIResearch tested three leading detectors, Pangram, GPTZero, and Originality, on both AI and human text
• On plain AI-generated text from basic prompts, false negative rates were near zero, at most 0.7%, detectors are now genuinely good at catching AI, a sharp reversal from a few years ago
• Originality had a 4% false positive rate on human writing, Pangram hit 0% false positives and 0% false negatives across 792 tested passages
• The gap: all three detectors do noticeably worse when AI is specifically imitating a real human author's writing style rather than just generating from a basic prompt
@theojaffee: "This is a total reversal from Claude 2 in 2023, which used to read as 100% human on GPTZero."
@schisofrenia: "It must be so annoying to be a professor grading an essay these days, because there's so many subtle ways you can obviously AI your way through it."
Epoch AI@EpochAIResearch
We stress-tested some AI detectors and found that they rarely flag human text as AI-generated. But asking LLMs to mimic a specific author causes detectors to misclassify text as human-generated ~13% of the time. For scientific writing, false negatives rose to ~26%.
English