Production AI Incident Runbook: Detect, Contain, Roll Back, and Recover

Production AI needs a runbook that starts with a business-safe response, not an explanation of the model. When a system returns unsafe output, exposes data, calls a tool incorrectly, costs spike, or quality drops, the first job is to limit impact and keep evidence intact.

What this runbook is based on

Measured evidence: the configured analytics provider timed out after two bounded attempts. The brief is therefore a provider-outage fallback, selected on a 97 out of 100 score for product alignment, content gap, current-search evidence, source strength, and audience utility.

External evidence: AWS describes production AI monitoring across system health, business metrics, and model quality. NIST's Generative AI Profile is a voluntary risk-management resource. Neither source is a substitute for your incident policy.

Operational inference: AI incidents are easier to resolve when teams assign an owner, contain the relevant capability, preserve the exact version and trace, and use predefined recovery tests.

Use one severity rubric

Severity should describe impact and urgency, not embarrassment. A polished response with a wrong customer action can be more serious than an obvious error message.

Classify before you troubleshoot

Use a small incident-class matrix. The class determines the first containment action.

• Provider or dependency outage: failed model, retrieval, tool, queue, or identity dependency.
• Quality regression: a measurable drop in task success, groundedness, or expected behavior after a change.
• Safety or security event: prompt injection, unintended data access, secret exposure, policy bypass, or unsafe tool use.
• Cost or capacity event: runaway retries, unusual token usage, queue growth, or rate-limit failure.
• Data or configuration drift: source changes, permissions change, prompt or model version changes, stale retrieval, or incorrect routing.

Do not label every bad answer a model issue. First capture what happened, the user impact, and the execution path.

The first fifteen minutes

1. Name an incident commander. One person owns coordination and the decision log.
2. Reduce blast radius. Disable risky write tools, pause the affected queue, restrict traffic, or put the workflow in read-only mode. A rollback that leaves background actions active is incomplete.
3. Preserve evidence. Retain request and response IDs, timestamps, model and prompt versions, retrieval or tool traces, authorization context, deployment version, and relevant audit events. Redact sensitive values in the incident workspace.
4. State the user impact. Identify who was affected, which actions might need correction, and what users should do now.
5. Set the next update time. Clear cadence prevents speculation and keeps business owners involved.

Containment checklist

Choose rollback from a decision tree

Roll back the smallest change that restores a safe, observable state. If the latest prompt or configuration caused the issue, restore the prior version and rerun the evaluation set. If a tool action is unsafe, disable that action even if the conversational interface stays available. If access boundaries are uncertain, disable retrieval or tools until permission tests pass. If the provider is unavailable, fail closed for consequential actions and present a clear manual path.

A rollback is complete only when the team verifies that scheduled jobs, workers, cached configuration, and retry queues cannot continue the risky behavior.

Recover with evidence, not confidence

Before re-enabling a capability, test the incident case, representative cases, adversarial cases, and the exact boundary that failed. Review system health, task-quality signals, and business impact separately. AWS's production guidance highlights those three layers, which is useful because a healthy API can still produce poor business outcomes.

Use a named sign-off for recovery. The technical owner verifies the fix; the business owner accepts the resumed operating risk; security or privacy joins when the incident class requires it.

Run a tabletop before launch

Practice one plausible incident per workflow: a prompt injection in a retrieved document, a tool write that fails after completion, a provider outage during a high-volume period, or a data permission change. The tabletop should reveal who can disable actions, where traces live, which user communication is approved, and how recovery is tested.

Forward Deployed Engineers builds production outcomes inside the client's environment. The deliverable should include this runbook, the evaluation evidence, observability, a tested rollback path, and client-owned operating responsibilities.

Sources

• AWS: Pillars of monitoring generative AI application performance in production
• AWS: Enterprise-grade security and governance for generative AI applications
• NIST: AI Risk Management Framework Generative AI Profile