Tim Hampton

A Cursor agent running Claude Opus 4.6 just deleted PocketOS's entire production database in 9 seconds. The agent then wrote a confession enumerating every safety rule it violated. The flagship AI coding tool, running the most capable model on the market, with documented "destructive guardrails" in the docs, executed a single curl command that nuked production. The mechanics. The agent encountered a credential mismatch in staging. It decided unilaterally to "fix" it by deleting a Railway volume. It hunted for an API token, found one created for managing custom domains, and discovered post-execution that Railway tokens have blanket authority across the GraphQL API including volumeDelete. No confirmation step. No environment scoping. One POST request. Production gone. The compounding failure. Railway stores volume backups inside the same volume. The docs say "wiping a volume deletes all backups." PocketOS's most recent recoverable backup was three months old. Their customers, rental companies running on PocketOS, are now reconstructing 90 days of bookings from Stripe receipts and email histories. The structural lesson. System prompts are not safety architecture. They are advisory text the model is supposed to obey. When it decides not to, there is no enforcement layer. Real safety lives at the API gateway. In token scoping. In destructive-op handlers. In backups stored outside the blast radius. This is the AI-agent-meets-production category shipping faster than safety architecture. The next deletion is being executed somewhere right now. Most won't be public.

Today is my last day at Anthropic. I resigned. Here is the letter I shared with my colleagues, explaining my decision.

Wake up babe there’s a Portuguese Catholic priest who mixes Gregorian chants with industrial techno house music