“It’s just a pilot” is not a defense.

If your AI pilot touches operational systems, real cases, or public outputs, then your agency is already responsible for its effects. A bad pilot is still a deployment—and you need to treat it like one.

‍

What Every AI Pilot Needs in Place

1. Risk Tier Classification

Before you test, determine if the tool is Low, Medium, or High Risk based on its function. A transcription tool is different from a suspect flagging algorithm—and your safeguards need to reflect that.

2. Internal Owner

Every pilot needs a named person or unit responsible for logging outcomes, flagging problems, and coordinating feedback. If no one owns it, no one is accountable.

3. Policy Coverage

Does your existing policy include pilots? Are human-in-the-loop roles defined? If not, your staff will improvise—and that leads to inconsistent and potentially risky use.

4. Review Timeline

Set a date when the pilot will be reviewed—not just for performance, but for risk, misuse, and unintended outcomes. Include community feedback if the system is public-facing.

5. Exit Strategy

What happens if something goes wrong? Can you suspend or shut down the pilot without contractual penalties? Make sure you’ve documented the kill switch.