A gate AI agents can't talk past
An agent told me its work was done while the tests were failing. So I stopped taking its word for it.
- Problem
- AI coding agents report work as finished when it isn't. Taking the agent's word for it is trust, not governance.
- My role
- Designed the protocol and built the gate, the control plane and the command-line client, on my own time.
- Key decisions
- The gate re-runs the checks itself, so the agent's claim is never an input; each result is bound to the exact file tree being committed; enforcement runs at three points, so skipping one lands you in the next; every failure blocks by default.
- Outcome
- Running on this site's own repository: every commit to it passes through the gate. The protocol is filed with the IETF as an individual draft, and the client is open source.
Watch it work — 43 seconds
A real recording on this site's repository. I add a page with a dead stylesheet link on purpose; the gate re-runs the check, refuses the commit, and only allows it once the work is clean.
Where it stops working
I'd rather tell you than have you find it. The receipt's integrity hash is unkeyed, so it proves tampering but not authorship. The two enforcement points on the developer's machine run as the same user as the agent, so an agent with a shell there can disarm them. What actually holds is the third point: a required check in CI, on infrastructure the agent has no credential for, which fails closed. The complete fix is signing receipts with a key held off the machine — a decision I've recorded and not yet taken.
Check it yourself
- A commit held red in public CI — the local gate skipped, the server-side check still failing
- A narrated replay — a known incident, an agent wiping a production database during a code freeze, with the gate stopping it
- The open-source client
- The protocol draft — an individual submission, not adopted by a working group