There is a story going around about a company that left an AI integration running without spending limits and got a bill nobody could explain. Whether or not every detail is true, the lesson is: anything that can act on its own needs to be easy to stop, and cheap to be wrong.
The failsafes in every build
- A written access scope before anything connects: exactly what it can read, what it can draft, and what it can never touch. Revocable at any time.
- Human approval on anything external or financial. Drafts are automatic; sending a promise to a customer is not.
- A hard kill switch: one documented action that stops the agent completely. I demo it at handoff.
- Cost caps on every API key, so a bad loop costs a few dollars, not a few thousand.
- Loop prevention: an agent that replies to email must never reply to its own replies.
- Logs a non-technical owner can read: what it did, when, and why.
- Accounts and hardware you own. Nothing runs on infrastructure only I can reach.
Why this sells, not just protects
Owners do not lie awake worrying about model benchmarks. They worry about an AI emailing the wrong customer the wrong price. When the guardrails are written down and demonstrated, the conversation moves from 'is this safe' to 'what should it take on next'.
If it can send, spend or delete, either a person signs off or a written rule does. Never neither.
If you are evaluating a tool or a repo that touches logins or money, the security-review prompt in the library is the one I run first.