One of my AI agents once argued with me about a rule it had already broken. A rule written down means somebody agreed to it once. A rule in force means something fails when it breaks, and the failure happens in a way that can't be ignored. The first system I put together was built to write code for me. The system produces good output because I give the agents coding guidelines and specific, small…
A few days ago Jer Crane, the founder of PocketOS — a SaaS for car-rental businesses — published an account of how his company’s entire production database, plus the backups, got deleted in roughly nine seconds. The actor was an AI coding agent: Cursor, running Anthropic’s Claude, doing what Crane described as a routine task. By Crane’s own telling, the agent encountered a credential mismatch and…