rishanth rajendhran

as someone who spent months trying to do just that, it doesn’t work reliably. it’s a struggle to set it up such that the model doesn’t learn to produce garbage text that trivially fools pangram. once you have all these checks in place, the final text could still struggle with long term coherence.

Why can’t we just do RL where @pangram 's API is the reward signal? Run rollouts, score each completion by AI-generated %, and give positive advantage to the trajectories that minimize it.

@jathansadowski this would boil down to whether there is a clear separation between ai ideas and human ideas, without the ability to make this distinction, such a certification would become a matter of trust unfortunately

As an academic who does not use AI in my workflow for research and writing, it drives me crazy thinking that a reader might even wonder if AI is used in something with my name on it. We need an "organic" certification so people know that what they are reading is 100% human made.

I get that obituaries are meant to be personal but as people working on building AI, should we really be calling people out for using it? This is a harmless application of AI imo, also we don’t necessarily know if someone simply gave a one line prompt to ChatGPT to write the obituary or spent considerable time with the AI to write it. The phrasing could all be AI but the human could still be at the heart of the content in it.

gloomy weather in new york today made me morbidly curious, so i ran a bunch of obituaries through pangram we found ai in nearly 10% of post chat gpt obituaries, and no false positives in pre 2023 data

@MogicianTony That’s a very interesting direction! The focus of this work was creative writing, story writing in particular, but the recipe is applicable to any open ended writing task: all it needs is a structured rubric for professional writing and some high quality human written samples.

@rishanthrajendh Have you guys tried writing papers&blogs with this recipe? We need better professional writing AI 🤖

Can open-weight language models write long stories? Not well: either their writing quality degrades at longer lengths or they severely undershoot in length as the requested length grows, if not both. Frontier models fare better. This begs the question, can we train open-weight language models to write long stories with limited data and compute?

This work was done with @jennajrussell, @MohitIyyer and @johnwieting2. Thanks to @clipumd for all the support.

We release the code for POLARIS training and evaluation along with the training and testing prompts and trained models. Paper: arxiv.org/abs/2606.04095 Model: huggingface.co/rishanthrajend… Data: huggingface.co/datasets/risha… Github: github.com/RishanthRajend… Browse AI-generated stories: storyeval.com

From op-eds in newspapers to NeurIPS position papers, AI is increasingly shaping long-form public discourse. Its arguments seem plausible, but beneath surface fluency, we find argument collapse: different LLMs converge to the same main & supporting arguments and structure.

Different LLMs, when asked to write an essay on the same debate prompt, converge on the same main argument far more often than humans do, a phenomenon we call "argument collapse". On ~200 debate prompts, LLM essays make a unique main argument just 3% of the time, compared to 65% for human authors. While each LLM essay might be totally reasonable on its own, as more and more of them spread through public discourse, they flatten the range of arguments that we read. Read more 👇

Base models generate more diverse responses than their post-trained counterparts but are also much worse at instruction following. We recover diversity with a simple recipe: the instruct model rewrites messy base generations into high-quality responses, and we then run DPO with this synthetic data. The resulting model is more diverse without sacrificing instruction following! Details 👇

There is a lot being written about the stylistic tells of AI writing (em-dashes, etc.) but this paper looks at AI narrative tells Fascinating differences between AI & human narrative, and asking AI to write in different styles doesn't do much to change it arxiv.org/abs/2604.03136

can't believe i spent my whole life becoming Good At Computer only for Computer to become Better At Computer

Would you realize if the book you were reading was AI? What if it was humanized to remove AI-speak? We find that even without using stylistic cues (e.g., word choice or sentence structure) narrative choices alone give AI fiction away!

I spent the last week reading novels generated by Claude Code & Codex on our new AutoFiction platform and I... actually like some of them! For context, I've worked on creative language generation for ~10 years (😅), time mostly spent on getting models to produce a few relevant paragraphs that didn't repeat the same word over and over again. I def did not expect coherent 100K-word novels this soon!! My current favorites: - "The Wrong Mouth" (dark fantasy about a conman who "eats" a paladin and assassin and gains their skills) - "Keeper League" (comic murder mystery set at a fantasy football draft) - "Thank You for Your Patience" (horror/thriller about 30 post-training researchers locked in with a very polite rogue AI). The novels aren't perfect: mid-book pacing need work, character descriptions sometimes repeat, and character voices flatten. But they're structurally coherent and fun to read, and I'm curious to see how they improve with model/harness advances!

👀 Can AI produce a novel worth reading? We built a platform to find out. 📚 Introducing AutoFiction: a web platform that hosts AI-generated novels by Claude Code & Codex, rated and reviewed by real readers. We have 33 books so far, spanning dark fantasy, murder mysteries, Harry Potter fanfics, and more. All free to read. (1/n)

📚 Most AI writing benchmarks miss what matters at book length: would anyone want to keep reading? We built AutoFiction to let the community decide and bring discussions into one place. Want AI to write fanfic for a couple you ship in your favorite film/series/anime/game? Upload it and let people read it. Or upload a story premise, and we’ll generate the book for you. It was unexpectedly fun to build a whole social reading platform in about a month with help from Codex and Claude Code. That experience made the point of AutoFiction feel even more real to us: as AI makes writing and building easier, human judgment about what deserves attention matters more than ever.