Note · Applied AI
A fleet of agents that audits itself
The short answer
The hard part of running many autonomous processes is not building them, it is knowing which ones are dead. A process that fails silently looks exactly like a process that has nothing to do, and both report success. The only durable answer is an audit that runs on a schedule, costs nothing, and is allowed to switch things off: it checks that each process serves a stated goal, produced something recently, and has not been failing quietly for weeks.
Liveness is not outcome
The most reassuring dashboard in this kind of system is also the most misleading: every process green, every run exiting zero. Exiting zero means the script finished, not that the work happened. Any process that matters needs an outcome assertion, a file that must exist, a row that must appear, a number that must move, and that assertion has to be able to fail. If it has never failed, it has not been tested.
The audit must be free
It is tempting to have a language model review the fleet, because a model writes a nicer summary. But monitoring runs on a schedule, and anything on a schedule multiplies: a small per-run cost times a fleet times a month is a real bill, and the first thing anyone cuts when monitoring gets expensive is the monitoring. Audits should be shell, a query, a script, deterministic and free, so nobody ever has a reason to turn them off.
Per-unit price times an autonomous loop is the failure mode
Any feature priced per PR, per review, per run or per investigation deserves one question before it is enabled: what volume does this multiply against? A system that opens a hundred pull requests a day turns a dollar per review into a hundred dollars a day. The dangerous version of this is not the price, it is a budget that only sends an email. A cap that notifies is not a cap. It has to be able to pause something.
Every process names its goal, or it dies
The audit question that removes the most waste is the least technical one: which goal does this serve? A process nobody can attach to an outcome is not neutral, it is cost plus risk plus noise in every report. Making the goal a required field, and deleting whatever cannot fill it in, keeps the fleet smaller than it wants to be.
Write the decision before the work, not after
Before any autonomous run, record what problem it addresses, what evidence justifies it, what would count as success, and who checks. This sounds like bureaucracy and is actually the opposite: it is what lets you grade the result honestly instead of reading the output and deciding afterwards that it was what you wanted.
Keep reading