A Failure Taxonomy for AI PMs: Classifying Every Way Your Model Breaks

A failure taxonomy is a shared map of how your model breaks, built on two axes: visibility (silent vs. loud) and cause (model, input, infrastructure, policy). Naming each cell — hallucination, timeout, refusal, low-confidence drift, and more — gives PM, design, and engineering one vocabulary for triage instead of three competing intuitions.

Quick Answer: Cross visibility (does the failure announce itself, or hide inside a confident-looking answer?) with cause (model, input, infra, or policy) to get eight-plus named failure types. Each cell demands a distinct UX response — you can't design one screen for all of them.

Most AI teams don't have a failure vocabulary — they have a Slack channel where someone posts "is this a bug?" and three people disagree. Support calls it a bug. Engineering calls it "expected model behavior." Design has no idea what to build because nobody agreed on what happened. A failure taxonomy ends that argument before it starts: name the failure mode, and the UX response, the severity, and the owning team are usually implied.

The Two-Axis Map: Visibility Crossed With Cause

The fastest way to classify any AI failure is to place it on two axes at once — a failure that's silent demands completely different UX than one that's loud, regardless of what caused it. Combine visibility with cause and you get a grid that covers nearly every incident report your team will file.

Visibility asks whether the user (or your monitoring) can tell something went wrong without inspecting the output closely:

• Loud failures announce themselves — an error banner, a timeout spinner that never resolves, a flat refusal. The user knows to distrust the output.
• Silent failures look fine on the surface — a confident, well-formatted, wrong answer. These are the ones that erode trust slowly, because the user acts on them before anyone notices.

Cause asks where in the pipeline the failure originated:

• Model — the model itself generated something false, incoherent, or overconfident.
• Input — the prompt, context, or retrieved data was ambiguous, incomplete, or corrupted.
• Infra — the request never completed: rate limits, timeouts, dropped connections, provider outages.
• Policy — the system correctly declined per a safety or business rule, and the "failure" is actually a guardrail doing its job.

Most teams only build UX for the loud row, because it's the row that generates support tickets. The silent row is where trust actually dies — and it's exactly the row a shared taxonomy forces you to design for on purpose, not by accident.

Why the Grid, Not a List

A flat list of failure types ("hallucination, timeout, refusal...") tempts everyone to treat each one as an isolated bug to patch. The grid format forces two questions per failure — is it silent or loud, and whose layer caused it — and those two questions are exactly what determines the fix. A silent model-cause failure needs a confidence signal; a loud infra-cause failure needs a retry pattern. Same list, very different engineering tickets.

Model-Cause Failures: Hallucination and Incoherence

Model-cause failures originate inside the model's own generation process, independent of what you fed it — the classic case is a fluent, confident answer that's simply false. These are the hardest to catch because nothing in the request signals danger; the failure is entirely in the output.

Hallucination (silent, model-cause) is a model generating plausible-sounding content — a citation, a statistic, an API method — that doesn't exist or isn't true. Stanford's HAI and multiple model-evaluation groups have repeatedly found that even strong frontier models hallucinate on a meaningful share of long-tail factual or citation-heavy queries, and the rate climbs sharply the more specific and obscure the ask. The UX fix isn't "make the model stop lying" — it's giving the user a way to verify without having to become a fact-checker themselves. Designing for hallucination as a failure mode <https://placeholder>_ — see the internal link below — covers the pattern in depth: source attribution, confidence framing, and "verify before you act" microcopy.

Incoherent or looping output (loud, model-cause) is easier to spot: repeated phrases, self-contradiction within one answer, or a response that trails into nonsense. Loud is actually the UX gift here — you can detect it with a simple repetition or perplexity check and short-circuit to a retry or an apology state, rather than showing garbage.

1. Hallucination needs attribution UX — show the user where a claim came from, or flag that it couldn't be sourced.
2. Incoherent output needs detection UX — catch the pattern server-side and retry silently before the user ever sees it.
3. Both need a fallback path — "I'm not confident in this" beats a wrong answer delivered with total confidence.

For the microcopy layer specifically — the exact words that accompany a hallucination-risk answer — see microcopy for hallucinated answers <https://placeholder>_, which is the applied companion to this taxonomy entry.

Input-Cause Failures: Garbage In, Confident Garbage Out

Input-cause failures happen when the model does exactly what it's supposed to with what it was given — but what it was given was ambiguous, stale, or wrong, so the output inherits the defect. These are diagnostically tricky because the model behaved correctly; the defect is upstream.

A stale or wrong retrieval (silent, input-cause) is the retrieval-augmented-generation failure mode: your vector search pulled an outdated document, or the wrong chunk, and the model faithfully summarized it. The output reads as authoritative because it's grounded in something — just the wrong something. This is arguably worse than hallucination for user trust, because the answer even has a citation attached.

A malformed query rejected (loud, input-cause) is the friendlier sibling — the system catches an ambiguous or incomplete prompt before generation and asks a clarifying question instead of guessing. This is the failure mode you actually want more of, because turning a silent failure into a loud one, earlier in the pipeline, is often the single highest-leverage UX intervention available.

Distinguishing input from model failures matters for triage: if the same prompt against a corrected context produces a good answer, the model is fine — your retrieval, context window, or data pipeline is the actual bug.

Infra-Cause Failures: When Nothing Comes Back At All

Infra-cause failures are the ones with the clearest UX playbook, because the entire discipline of latency and error-state design in software predates LLMs by decades — the only new wrinkle is that AI responses are slower and less predictable than a typical API call. A timeout or a rate-limit error simply means the request never completed; nothing was generated to evaluate.

The design principle across this whole row: never let an incomplete response look like a complete one. A stream that cuts off mid-sentence and just... stops, with no visual marker, reads to the user as the model finished and simply had nothing more to say — which is a much worse trust failure than the actual outage. Jakob Nielsen's original heuristics on system status visibility apply here almost unchanged; AI just raised the stakes on getting it right.

Policy-Cause Failures: When the Refusal Is the Feature

Policy-cause failures are the one category where the "failure" is arguably not a bug at all — the system is doing exactly what its guardrails were designed to do, and the design challenge is making that legible rather than making it go away. Conflating this row with genuine bugs is one of the most common sources of the "is this a bug or expected?" argument this taxonomy exists to end.

An explicit refusal (loud, policy-cause) declines outright — "I can't help with that." This is the easy case: loud, unambiguous, and if the refusal reason is stated clearly, defensible.

A soft-declined answer disguised as a normal one (silent, policy-cause) is the dangerous variant: the model hedges, deflects, or gives a watered-down non-answer that reads like a real one unless you're looking for it. Users mistake evasion for a genuine attempt and act on incomplete guidance. The UX fix mirrors the hallucination fix: make the boundary visible rather than letting it hide inside fluent prose.

• Explicit refusals should state the reason in plain language, not a generic error code.
• Soft declines should be detected and re-labeled — "I can partially help, here's what I can't cover" beats a vague non-answer.
• Policy failures should never be logged the same way as model bugs — conflating the two corrupts your incident metrics and makes your guardrails look like defects.

Why This Taxonomy Ends the "Bug or Expected?" Argument

The recurring failure mode in AI product teams isn't any single error type — it's the triage meeting where support, engineering, and design each assume a different definition of "broken." A shared taxonomy fixes this because visibility and cause are both observable facts, not opinions: you can check the logs and know within seconds which cell an incident belongs in.

Once a failure has a name and a grid cell, three things follow automatically:

1. Ownership is implied. Infra-cause failures route to engineering; policy-cause failures route to whoever owns the guardrail policy; model-cause failures need a UX or prompt-layer response, not a code fix.
2. Severity is implied. Silent failures are typically higher-priority than loud ones of the same cause, because they've already reached the user unflagged.
3. The fix category is implied. You don't debate whether to build a confidence indicator or a retry button — the cell tells you which.

This is the same maturity move product teams already made with incident severity levels (SEV1-4) and with root-cause categories in reliability engineering — Google's SRE discipline formalized exactly this kind of shared taxonomy for infrastructure outages decades before LLMs existed. AI failure classification is the same idea applied to a fuzzier, higher-stakes surface: the model's judgment itself.

Where Prodinja Fits

Prodinja's UX of Failure tool starts from a version of this same idea: it organizes failure patterns around four core modes — Hallucination, Timeout, Refusal, and Low-confidence — as a ready starting taxonomy rather than a blank page. It's designed as a prototype workspace for mapping which of your product's actual failure states fall into which mode and sketching the UX response for each, extending the four starting modes with the input- and policy-cause cells this article adds as your product's specific failure list grows.

The point isn't that four modes are the final answer — it's that starting from a named structure, rather than an unstructured bug list, is what makes the taxonomy exercise finishable in an afternoon instead of an open-ended debate.

Key Takeaways

• Cross visibility with cause — silent-vs-loud and model/input/infra/policy — to get a grid that covers nearly every AI failure your team will encounter.
• Silent failures are the highest-trust-risk category because they look like normal, confident output and get acted on before anyone flags them.
• Hallucination and incoherence are both model-cause, but one is silent and one is loud, so they need opposite UX treatments: attribution versus detection-and-retry.
• Input-cause failures mean the model behaved correctly on bad data — the fix belongs in retrieval or context, not in the model layer.
• Infra-cause failures are solved by decades-old status-visibility principles — never let an incomplete response look complete.
• Policy-cause "failures" are often guardrails working as intended — the design job is making the boundary legible, not eliminating it.
• A named taxonomy converts a triage argument into a lookup — once a failure has a cell, ownership, severity, and fix category typically follow.

Frequently Asked Questions

What is an AI failure taxonomy?

An AI failure taxonomy is a structured classification of the distinct ways an AI system can produce a wrong, incomplete, or unhelpful result, organized by traits like visibility and cause. It replaces an ad hoc bug list with a shared vocabulary that product, design, and engineering can all reference during triage.

What are the main types of AI errors in a product?

The main types split into model-cause (hallucination, incoherent output), input-cause (bad retrieval, malformed queries), infra-cause (timeouts, rate limits), and policy-cause (refusals, soft declines) failures. Each type further splits into silent or loud depending on whether the failure is visible without inspection.

How do you classify LLM failure modes for a product team?

Classify each observed failure along two axes at once: is it silent (looks normal) or loud (announces itself), and did it originate in the model, the input, the infrastructure, or a policy guardrail. Most teams find eight to twelve concrete failure types once they map their actual incident history onto this grid.

Is a refusal always a bug?

No — an explicit refusal is frequently a guardrail functioning correctly, not a defect, and should be tracked separately from genuine model or infra bugs. The real risk is the soft-decline variant, where the system hedges in a way that reads as a normal answer instead of a visible refusal.

Why does silent versus loud matter more than the root cause?

Visibility determines how much damage a failure does before anyone notices, which is often more urgent than knowing exactly why it happened. A silent hallucination and a silent bad-retrieval answer look identical to the user and demand the same immediate UX fix — visible attribution — even though their root causes and engineering fixes differ.

Building this taxonomy pays off further once you connect it to how users actually experience these failures across a session — the Customer Journey <https://placeholder>_ emotion curve and the Jobs to Be Done <https://placeholder>_ framing both help locate exactly where in a user's task a given failure mode does the most damage, and the complete guide to UX of failure <https://placeholder>_ is the fuller reference for turning this taxonomy into a shipped design system.