Stop writing AI policies. Start writing AI runbooks.

By now your organization almost certainly has an AI policy. It was probably drafted by a cross-functional working group, reviewed by legal, and published to the intranet with an all-hands mention. It says the right things: AI use shall be responsible, output shall be verified, human oversight shall be maintained, data shall be protected.

Now run a small thought experiment. It's 2am and an autonomous agent in your order pipeline has started issuing actions outside its defined workflow. Who gets paged? What do they do first? What do they shut off, and how? What evidence do they preserve before state gets overwritten?

If the answer lives anywhere, it doesn't live in the policy. The policy says human oversight shall be maintained. It does not say which human, reached how, empowered to do what. At 2am, "shall be maintained" is not a phone number.

Policies describe intentions. Runbooks describe actions.

This is not an argument against policies — you need them, and regulators and auditors will ask for them. It's an argument about sequencing and where the marginal hour goes. Most organizations are now on revision three of the policy while owning zero runbooks, and the security discipline already learned why that ratio is backwards.

A security policy that says "vulnerabilities shall be patched promptly" is compatible with any behavior at all, including none. What made patching real was never the policy sentence — it was the operational apparatus underneath: severity tiers with SLAs, a named owner per system, an escalation path, and a procedure someone can execute at 2am without convening a committee. Policy is what you believe. Runbook is what you do, written down before you're doing it under pressure.

AI operations is at the stage security was at when it had beliefs and no procedures. The failure modes are already known — agents deviate, vendors ship silent changes, injections land, models get swapped upstream — and almost nobody has written down what to do when each one happens.

Five runbooks worth writing before the next policy revision

Each of these earns its keep the first time it's used. Each fits on a page: trigger, owner, first steps, evidence to preserve, escalation.

1. A third-party AI surface changed. A vendor's MCP server added a tool; an integration widened its schema; a model version rolled. Who is notified? Who decides whether the change is acceptable, on what criteria, within what window? What happens to the connection in the meantime — freeze, monitor, or trust by default?

2. An agent deviated from its workflow. Skipped an approval step, acted out of sequence, exceeded its threshold. Who gets the alert? Is the agent paused automatically or by a human call? What record of the run gets preserved before anyone touches anything? Who tells the process owner whose approvals were bypassed?

3. Suspected prompt injection. Output suggests the model followed instructions that came from data — a document, a ticket, a peer agent's message. First steps: isolate the input source, preserve the full context window, inventory which other systems consumed the same source. Decide: is this an ongoing exposure or a one-shot?

4. An upstream model or vendor incident. The provider announces degraded behavior, a security issue, or an unannounced behavior change breaks your workflows. Which of your systems depend on that provider — can you answer from an inventory, or does answering become the incident? What's your fallback: alternate model, degraded manual mode, pause?

5. An employee reports AI-caused harm to a customer. Wrong information sent, wrong action taken, wrong record changed. Beyond service recovery: who determines whether it's an isolated output or a systematic behavior that has been repeating quietly? What does the disclosure obligation look like, and who owns that call?

The tooling test

Writing these runbooks delivers a second benefit, arguably the larger one: every step you cannot execute exposes a control you don't have.

Runbook 1 says "review the diff of the vendor's change" — but nothing captures snapshots of the vendor surface, so there is no diff to review. Runbook 2 says "preserve the record of the run" — but the agent produces no durable run record. Runbook 4 says "consult the dependency inventory" — which doesn't exist. Each dead step is a precise, prioritized gap statement: this signal, this record, this capability is missing, and here is the incident scenario where its absence hurts.

That makes runbook-writing the cheapest AI risk assessment available. A policy revision produces prose that is compatible with everything. An afternoon of runbook drafting produces a concrete backlog of operational gaps, ranked by how badly you'd miss each one at 2am.

The policy tells the auditor what you believe. The runbook tells the responder what to do. When something eventually goes wrong — and in every other domain of computing, it eventually does — nobody in the incident channel will paste in the PDF. Write the page they will paste.