The system failed. Now what?

If you don’t have an answer to that question, you’re not ready for operational AI.

AI systems can fail in ways that catch agencies off guard: a misleading output, a hallucinated alert, a pattern-matching error that leads to a bad outcome. Maybe the press hears about it. Maybe the oversight board does. Maybe it ends up in court.

When that happens, it’s not your vendor who’s on the hook. It’s your agency.

‍

Five Things That Go Wrong When There’s No Plan

1. There’s no kill switch

The system keeps running—even as your team scrambles to understand what it did. That’s operational and legal exposure in real time.

2. There’s no audit trail

You can’t say what inputs went in, what outputs came out, or who reviewed them. Without logs, you can’t defend decisions—or correct course.

3. There’s no policy for incident response

The policy you do have doesn’t mention how to handle public complaints, field misuse, or suspected bias. It’s not operational—it’s shelfware.

4. The vendor goes silent

You realize your agency has no contractual right to pause, audit, or investigate the tool. And the vendor’s PR team just posted a blog saying everything is fine.

5. The press gets ahead of you

Before your team meets, the story’s already out. Now you're answering questions without documentation.

What a Prepared Agency Has in Place

• An incident reporting form used by staff
• A review protocol with assigned roles
• A kill switch clause in every contract
• A public communication workflow
• A quarterly review process for all high-risk AI tools