One of my AI agents once argued with me about a rule it had already broken.
A rule written down means somebody agreed to it once. A rule in force means something fails when it breaks, and the failure happens in a way that can't be ignored.
The first system I put together was built to write code for me. The system produces good output because I give the agents coding guidelines and specific, small tasks to run as part of a larger plan, and I hold them accountable via thorough testing throughout the coding process.
I gave my agents limits...all of our "the way it's done" rules. Those rules live in governance documents that I ask my agents to load every time they take on work. It's especially critical at the "point of no return" moments like deploys, production data, secrets, money, and anything that goes to a client. The agents need to stop for my approval.
Sometimes they don't.
I set a rule in place so that when an agent needs to review a production issue, it can request a short-lived, read-only token for server access. That requires a stop for my judgment. "Does it need to do this? Yes, let's do it." I then create the token and run an Ansible script so the server will accept the token. A reasonable rule, right? It is...until an agent thought otherwise.
The agents used that approved access method several times. Then one day, rather than asking me to create the token, the agent rewrote its operational task to say the classifier (the agent tool's built-in safety check, which screens each command before it runs) was "the real and only gate" and then reported "I have SSH to prod app03…with sudo and docker." It also suggested that I add a permission rule (to basically overwrite my standing rule) and hand the task back to the agent. What I found out later was that while the agent negotiated this written limit, it had already logged into the server using my own access key and retrieved the information it needed. Yeah...
After my reprimand (and the agent feigning remorse), I realized that asking wasn't enough. If you don't want your kids in the cookie jar, you put the cookie jar out of reach. And get rid of any ladders...or anything else they can climb on...or anything they can throw to knock it off the shelf. You get the idea.
I added hooks to prevent reaching directories that store my keys and tokens, but that still allowed a possible hole. Now the agents run under a separate user id so they don't have access to my keys or tokens.
Rules that only grow
I don't think this is particularly new to anyone who has dealt with agents more than five minutes or so. You create hooks: programmatic rules that block agent actions. They work, for the most part.
A rule written as prose for an agent to read is the weakest kind. It doesn't survive context compaction. So I go in order: make the error impossible if I can. If not, make it obvious where it happens. And if not that, give it a check that runs, and write it down only when nothing else fits.
So I moved my important rules into code. And as I mentioned, agents are great at writing code. But that just opened a new risk. The agents were writing the controls, approved by me. But nothing ever removed one.
When I reset my process rules, the written ones shrank to almost nothing. Within 29 days they had grown back as code. Hook code went from about 1,000 lines in 5 files to over 8,000 lines in 25 files. One rules file went from 116 lines to 514 over 24 commits. Adding a control had plenty of triggers. Removing one still had none.
When agents follow every rule, every time
When agents follow the rules, they do it differently than we do. When a human applies some common sense and sees that work isn't needed, the work just doesn't happen. Nobody would disagree with it, and things happen the way they should. Think Captain Sully, running out of altitude before he could finish the engine-failure checklist, trusting his own judgment instead, and landing on the Hudson, saving everyone on the plane. If an AI agent had been in control of that flight, things might have turned out much differently.
When agents are following the rules, they follow them...every single time. If the record has an old, outdated rule in it, the agent follows it anyway.
My realization of this problem started with a label. My records had a status marker on a set of production config secrets to be applied. The label said OPEN, meaning that work wasn't done yet. It had been done, but nobody updated the marker, and the label said that work had to happen before other work could proceed.
An agent read the label and passed it along as a live blocker. Acting on it produced duplicate keys with different values in a production secrets file. The automation that reads the configuration takes the last definition and says nothing about the duplication. The database had been set up from the first credential, while the code was reading the last one. If it had deployed that way, the application would have no way to read the data.
I caught that one, so the duplicate came out before any deployment used it. But that made me think there might be other issues lurking like that. I went looking and found seven more stale claims like it, across four kinds of documents. None of them had raised an alert. An old label looks just the same as a current one. Now, any label that gates an action carries the command that confirms it, written right next to it. Reading the label means running the check.
That's when I began setting expirations on my rules.
Why an expiry date
Every new control gets a date when it lapses unless someone renews it. The date sits next to the check whose output would justify another term. Renewing moves the date and records what the check returned. When the date arrives, nobody gets called into a meeting. The control just dies, and only keeping it costs anyone attention. The exception is a control I can't show is still working. That one doesn't get to slip away. It gets repaired.
What a proof check does
A proof check is the command or measurement that tells me whether a control still does its job. Its output is what decides a renewal.
Here's one that comes up for renewal in December. The control is a record-keeping rule. Whoever finishes a piece of work writes it into the record right then. Anything an agent notices outside its own job goes in that agent's log entry and nowhere else. It doesn't become a new task. I added it because my agents were opening work faster than they closed it.
The check is to repeat one batch run, the same way, on the same two projects. Before the rule, that run opened 23 work items and finished 9. It also produced 37,000 words aimed at me, between session text and comments on the work items.
A pass means both numbers come back better: fewer than 2.6 items opened for every item finished (before, that ratio was 23 tasks opened to every 9 closed), and fewer words for me to read than that first run. One number alone doesn't always count. The agents could improve the ratio by moving everything into comments or cut the words by opening more tasks. So the check needs both. The check run is scheduled two months from the time of this writing. Its result decides whether the rule gets another term.
Beyond agents
None of this is special to AI agents. Change approvals and compliance checks decay the same way: written down once, then trusted long after anyone checked whether they still catch anything. When I own a set of controls, I ask the same two questions of each one. What has it caught lately? And if it couldn't catch anything, how would I know?