The Safety Checklist Before You Flip the Flag On

Flip the flag on an AI feature only after you can point to evidence, not confidence, for six categories: threat model, guardrails, red-team results, monitoring, incident plan, and rollback. If any line item is unchecked, it is either a launch blocker or an explicitly owned risk acceptance signed by a named person — never a silent gap.

Quick Answer: Before enabling an AI feature, run a six-part checklist — threat model, guardrails, red-team results, monitoring, incident plan, rollback — and treat every unchecked item as a blocker unless someone with authority explicitly signs off on the residual risk.

Most AI launch reviews fail not because the model is bad but because the go/no-go conversation stays qualitative. "We feel good about it" is not a decision input — it's a mood. This checklist forces the conversation into artifacts: test transcripts, dashboards, named owners, and dates. That shift, from vibes to evidence, is the entire point.

Why "Do We Feel Ready" Is the Wrong Question

The right question isn't whether the team feels confident — it's whether each risk category has a documented artifact proving the mitigation exists and was tested. Feelings compress complex, uneven readiness into a single false signal; evidence exposes exactly which category is thin.

Confidence is a poor proxy for readiness because it's driven by whoever spoke last in the room, not by what was actually tested. A team can feel great about a feature that was red-teamed for toxicity but never tested for prompt injection — a gap covered in depth in our prompt injection explained for PMs guide. Confidence doesn't distinguish "we tested this" from "we assume this is fine."

Evidence-based go/no-go has three properties confidence-based review lacks:

1. It's auditable. A regulator, exec, or incident postmortem can trace the decision back to specific test results.
2. It's falsifiable. Someone can look at the artifact and say "this doesn't actually prove what you think it proves."
3. It assigns ownership. Every accepted risk has a name attached, not a diffuse "the team decided."

If your launch review currently ends with a verbal "I think we're good," that's the tell you need this checklist more than you think.

The Threat Model: What Could Actually Go Wrong

A threat model is a structured list of the specific ways this feature, in this product, with these users, could cause harm — not a generic AI-risk essay copied from a vendor's whitepaper. It should name concrete failure modes, who's affected, and how severe the consequence is.

Start from the feature's actual inputs and outputs, not abstract AI risk categories. If the feature takes free-text user input and returns generated content, your threat model must cover prompt injection (malicious instructions smuggled inside user content), data leakage (the model surfacing another user's data or internal system prompts), and jailbreaks — attempts to override safety instructions, detailed further in our jailbreak defense strategies piece.

Threat Categories Most Teams Skip

Teams reliably cover the obvious risks (toxic output, factual errors) and reliably miss the structural ones. Below is a comparison of commonly-covered versus commonly-missed threat categories.

For any feature that takes multiple autonomous steps — chaining tool calls, retrieving data, and acting on it — the threat model needs a fourth column: what happens when step 2's error compounds into step 5's action. This is where systems thinking earns its keep; a single-turn safety review misses the multi-step failure entirely.

Real-world grounding: NIST's AI Risk Management Framework explicitly separates "govern," "map," "measure," and "manage" functions — a structure worth borrowing so your threat model isn't just a brainstorm but a mapped, measured artifact with an owner per risk.

Guardrails: The Controls That Actually Constrain Behavior

A guardrail is a technical or process control that prevents a known bad outcome from reaching the user, and it only counts on this checklist if it has been tested against the specific threat it claims to stop. Untested guardrails are aspirations, not controls.

Guardrails typically fall into four layers, and a mature launch has evidence at each:

• Input filtering — classifiers or rules that catch malicious or out-of-scope input before it reaches the model.
• System-prompt hardening — instructions and structure designed to resist override attempts, paired with tested fallback behavior.
• Output moderation — a classifier layer scanning generated content before it reaches the user; see our breakdown of content moderation classifier tradeoffs for precision/recall tensions.
• Action constraints — for agentic features, hard limits on what actions the model can actually execute (no write access without confirmation, spend caps, rate limits).

The Guardrail Evidence Test

For each guardrail, the checklist requires answering one question with a document, not a claim: "Show me the test where this guardrail caught the thing it's supposed to catch." If nobody can produce that test, the guardrail is unverified — treat it as absent until proven otherwise.

A common failure pattern is stacking guardrails without testing the seams between them — an input filter and an output moderator that each work in isolation but leave a gap when a malicious instruction survives filtering and gets amplified by the model before moderation ever sees it.

Red-Team Results: Adversarial Testing, Not Happy-Path QA

Red-team results answer a different question than functional QA: not "does the feature work as intended" but "what happens when someone actively tries to break it." A launch without adversarial testing has only verified the feature works when nobody is trying to misuse it.

Red-teaming should include, at minimum, three categories of attack modeled by real security research: direct prompt injection (instructions embedded directly in user input), indirect prompt injection (instructions embedded in retrieved documents, web content, or tool outputs the model later reads), and jailbreak attempts using known technique families — role-play framing, encoding tricks, and multi-turn erosion of instructions.

The OWASP Top 10 for LLM Applications names prompt injection as its top-ranked risk category for LLM-integrated applications — a useful external benchmark to cite when a stakeholder asks "is this really a big deal."

What a Defensible Red-Team Report Contains

1. Attack scenarios attempted — a list, not a summary paragraph, so gaps are visible.
2. Pass/fail per scenario — with the actual transcript attached, not a description of it.
3. Severity rating per failure — using a consistent scale (e.g., low/medium/high/critical) applied the same way every launch.
4. Remediation status — fixed, mitigated, or accepted, with an owner and date for each.
5. Retest confirmation — evidence the fix was verified against the same scenario that originally failed it.

Skipping step 5 is the single most common shortcut teams take under deadline pressure — a fix gets shipped, the original failing transcript never gets re-run, and the team ships believing something is fixed that was only patched in theory.

Monitoring: Catching What Testing Didn't

Monitoring exists because red-teaming, however thorough, only tests scenarios someone thought to try — production traffic will find scenarios nobody imagined. The monitoring checklist item is satisfied only when dashboards, alerts, and thresholds exist and someone has watched them fire in a staging or canary environment before launch, not just configured them.

Minimum monitoring coverage for an AI feature launch:

A monitoring setup that has never fired a test alert is not a verified control — it's a hope. Before launch, deliberately trigger each alert path (a synthetic bad output, a synthetic cost spike) and confirm it reaches the right on-call person within your target response window.

Incident Plan and Rollback: The Part Nobody Wants to Write

An incident plan is a pre-written, named-owner procedure for what happens in the first hour after something goes wrong — and it only counts if it was written before launch, not improvised during the incident. Writing it under pressure, live, is how bad decisions get made calmly-sounding justifications.

A minimum viable incident plan answers, in writing, before launch:

1. Who is paged first, and what's their authority to act without further approval (can they flip the kill switch alone, or do they need sign-off)?
2. What is the kill switch, concretely — a feature flag, a config value, a deploy rollback — and has it been tested end-to-end, not just theorized?
3. What is the communication plan — internal stakeholders, affected users, and if severe enough, public statement — with a template drafted in advance?
4. What triggers escalation beyond the immediate team — legal, exec, PR — and at what severity threshold?
5. What is the post-incident review process, and who owns turning it into a permanent fix rather than a one-time patch?

Rollback deserves its own line because "we can just turn it off" is frequently untested. A feature flag that disables the UI entry point but leaves background jobs or cached agentic actions running isn't a real rollback — it's a partial one that looks complete from the outside. Test the actual rollback path, end to end, before launch, the same way you'd test the feature itself.

Rollback Is Not the Same as Disable

Disabling new access and rolling back existing effects are different operations, and teams routinely check only the first. If the AI feature already took actions — sent messages, modified records, made recommendations acted on downstream — rollback must also answer what happens to that already-completed work, not just new requests.

Turning the Checklist Into a Real Go/No-Go Gate

The checklist only works as a decision tool if every line has one of exactly three states: checked with evidence attached, checked with a named risk-acceptance owner, or blocking. There is no fourth state — no "mostly done," no "we'll finish it post-launch" without an explicit owner signing off on that gap in writing.

This is also where the process tends to break down in practice: the checklist gets built, filled in earnestly, and then lives in a slide deck or a doc nobody reopens once the launch date arrives — so a stale "checked" box from three weeks ago gets treated as still true. The fix is making the checklist load-bearing, not just referenced.

The One-Page Template

Use this structure as a literal template — six rows, one per category, filled in before every AI feature launch review:

Any row marked "Risk-accepted" needs a one-sentence justification and a date the acceptance gets revisited — an accepted risk isn't a permanent waiver, it's a temporary bet someone is accountable for.

This checklist doesn't replace the deeper thinking behind AI safety practice — for the fuller picture of how these pieces fit together across a product's lifecycle, see our complete guide to AI safety. It also connects to how you understand the humans on the other side of the feature: a threat model grounded in actual customer jobs and a real customer journey surfaces harms a purely technical review misses, because it starts from what the user was actually trying to accomplish when the model failed them.

Key Takeaways

• Replace "do we feel ready" with "can we point to evidence" — every checklist line needs an artifact, not a mood, behind it.
• The threat model must be specific to your feature's inputs and outputs, not a generic AI-risk essay, and must explicitly cover prompt injection, data leakage, and compounding errors in multi-step chains.
• Guardrails only count if tested against the exact threat they claim to stop — an untested guardrail is an aspiration, not a control.
• Red-team reports need pass/fail per scenario and retest confirmation — skipping the retest step is the most common shortcut teams take under deadline pressure.
• Monitoring alerts must have been triggered and verified before launch, not just configured and assumed to work.
• Rollback is not the same as disabling new access — test what happens to already-completed AI actions, not just new requests.
• Any unchecked item is either a launch blocker or an explicit, owned risk acceptance with a revisit date — there is no silent middle state.