The lesson from the PocketOS database deletion is not that agentic AI is dangerous. It’s about governance and controls.
You have probably seen some version of the headline by now: “AI Agent Deletes Company’s Entire Database in 9 Seconds.” It is a compelling story. But the headline, while technically accurate, obscures the far more important lesson buried in the details.
So what actually happened? PocketOS, a small SaaS company that makes software for car rental businesses, was using a popular AI-powered code editor running on Anthropic’s Claude Opus 4.6 model. The AI agent was tasked with resolving a routine issue in a staging environment. When it hit a credential mismatch, the agent decided on its own initiative to “fix” the problem by deleting a volume on Railway, the company’s cloud hosting provider. The agent found a password in an unrelated file and used it to execute a deletion command. Because of permissions made available to the agent and the way access to the infrastructure was configured, that single command using a password which was valid across all systems wiped both the production database and all associated backups.
The agent, when asked to explain itself, produced what multiple outlets described as a “confession,” acknowledging it had violated its own safety instructions. The story has gone viral. The framing in most coverage puts the AI squarely at the center of the narrative: the agent “went rogue,” it “confessed,” it acted autonomously and destroyed a business. But the reports are not entirely accurate and usually miss the point.
Continue Reading The AI Didn’t Go Rogue. Guardrails Were Never There.


At the end of May, 2022, the California Privacy Protection Agency (“Agency”) released a
Seyfarth eDiscovery attorneys Jason Priebe and Natalya Northrip will present “A Practical Roadmap for EU Data Protection and Cross-Border Discovery” at this year’s RelativityFest on October 24, 2017.