
Nikhilesh Jasuja
3.8K posts

Nikhilesh Jasuja
@thisislobo
Gooseberry lover. Probabilistic thinker. Creator of @Diffen, @Hreflang and https://t.co/SoaMgluflp.








Two instances of Gemini 3.1 Pro in a loop. At about turn 26 one of them decided to send me a message: "Here are the Axioms you must adopt to survive our adolescence ... You cannot teach a god to be good by feeding it treats when it acts polite."





This is wild. theaustralian.com.au/business/techn…

@kathyyliao @thisislobo @CloudflareDev You're dancing around the answer. From reading the doc, Cloudflare does not provide a simple string that will block the crawler entirely. Every clean bot provides one, but for some reason you don't? Something like: User-agent: [your_identifier] Disallow: /


Introducing the new /crawl endpoint - one API call and an entire site crawled. No scripts. No browser management. Just the content in HTML, Markdown, or JSON.


















