A streaming AI response can die before the first token, mid-sentence, or one word from the end — and each moment needs a different recovery: clean retry, preserve-and-mark, or resume-in-place. The core danger is that users read partial text as settled fact the instant it appears, so an unmarked half-answer left on screen reads as a complete, authoritative one.
Quick Answer: Design three distinct recovery states for streaming failure — before-first-token (retry cleanly, no partial artifact), mid-stream (preserve what rendered but visibly mark it broken), and near-complete (offer resume, not restart). Never leave truncated output on screen without a signal that it didn't finish.
Why streaming failure is a different design problem than request failure
Streaming failure is harder than request failure because the user has already started trusting a partial answer before your system knows it's broken. A traditional request either returns or errors — there's no in-between state a user can misread. A stream renders token by token, so failure can land anywhere inside an answer the user has already started reading as truth.
This matters because reading is believing in a token-streamed UI. Cognitive research on how people process incrementally-revealed text (going back to eye-tracking work on reading comprehension) shows readers build a mental model as each clause lands, not after the full passage arrives. A stream that stops after "The recommended dosage is 400mg twice daily, unless the patient" has planted a complete-sounding claim and then vanished mid-qualifier — and the reader's mental model doesn't automatically un-commit.
Three properties make streaming UX distinct from static request/response design:
- Partial output looks finished. A sentence with correct grammar and no visible cursor reads as complete even when it's cut off before a critical caveat.
- The failure boundary is invisible by default. Unless you design a signal, there is no visual difference between "the model said this and stopped" and "the model was cut off."
- Recovery options differ by how much already rendered. Restarting after 2 tokens costs nothing; restarting after 400 tokens throws away work the user already read and may have acted on.
Confidence displays are a related but separate concern — even a complete answer needs honest signaling about certainty, as covered in confidence displays without scaring users. Streaming failure UX is about the mechanical interruption, not the model's epistemic confidence, though the two often need to cooperate on screen.
The anti-pattern: the authoritative half-answer with no signal
The single worst pattern in streaming UX is a truncated response that ends on a period, a comma, or mid-word, with no cursor, no error banner, and no retry affordance — it just stops. The user has no way to distinguish "that's the whole answer" from "the connection dropped."
Anti-pattern check: if a screenshot of your failed stream, taken cold with no context, could pass as a complete and correct answer, you have this bug.
This happens most often because the failure is caught below the rendering layer — a fetch abort, a socket timeout, a provider-side 500 mid-stream — and the UI code has no hook to react, so the last rendered frame simply persists. Three concrete failure modes feed this anti-pattern:
| Cause | What the user sees without a fix | Why it's dangerous |
|---|---|---|
| Network drop mid-stream | Text stops at whatever token last arrived | Reads as a complete, confident sentence |
| Provider rate limit / 5xx after partial tokens | Same as above, sometimes with a trailing partial word | User may act on incomplete numeric/instruction content |
| Client-side timeout abort | Stream frozen, no further updates, tab still shows a spinner or nothing | User waits indefinitely or assumes success |
The fix is never clever copy alone — it's a state machine that always distinguishes "done" from "stopped." Every streaming client needs an explicit terminal state (complete, interrupted, errored) that the UI renders differently, so "no more tokens are coming" always has a visible cause attached to it. This is the same discipline designing for the hallucination failure mode argues for confident-but-wrong content — a UI should never let a failure mode look identical to success.
Interruption point 1: before the first token
Failing before the first token is the cheapest failure to recover from — nothing has rendered yet, so a clean retry costs the user nothing but time. The design job here is entirely about perceived latency and honest waiting, not about salvaging partial content.
Common causes at this point: cold model load, auth/rate-limit rejection, or a queue timeout before the stream opens. Because there's no partial text to protect, the interface can be blunt:
- Show a distinct "waiting for response" state (not a generic spinner reused everywhere) so a slow-start failure doesn't get confused with a mid-answer stall later.
- Set an explicit timeout threshold (commonly 5-15 seconds depending on your model's typical time-to-first-token) after which you surface a retry, not an indefinite spinner.
- On failure, offer a single clean Retry action — same prompt, fresh request — with no partial artifact to reconcile.
- If the failure is systemic (provider outage) rather than transient, say so plainly rather than implying the user's specific prompt was the problem.
The mistake to avoid here is treating "no first token yet" and "stream that started then died" as the same UI state. They call for opposite framing: the first is "hasn't started, try again"; the second — covered next — is "started, then broke, here's what we have."
Interruption point 2: mid-stream, preserve and mark
A mid-stream break needs the opposite instinct from a pre-first-token failure: preserve what already rendered, and mark it, rather than discard or silently freeze it. The user has already read part of the answer — throwing it away and starting over wastes their attention and can feel like the system is hiding something.
The direct answer: keep the partial text visible, append an unambiguous interruption marker at the exact cutoff point, and pair it with a Continue or Retry from here action rather than a full restart button.
Concretely, a well-designed mid-stream failure state includes:
- A visual break marker at the exact point streaming stopped — a horizontal rule, a dashed underline, or an icon inline with the text, not just a banner below unrelated to the cutoff location.
- Explicit copy naming what happened: "Response was interrupted" or "Connection lost while generating" — never a silent stop.
- A recovery choice, not just an acknowledgment:
Continue generating(resume from the cutoff) where your architecture supports it, orRetry(regenerate from scratch) where it doesn't. - No implied completeness — remove any trailing punctuation styling, cursor blink, or checkmark that would visually suggest the sentence finished on its own.
This is where preserve-and-mark as a pattern differs sharply from a plain retry: you are explicitly telling the user "this much is real, this much is missing," which is a fundamentally different trust contract than "ignore that, here's a new answer." Product teams that skip this step and just silently re-run the whole generation on retry often see users lose confidence in the tool faster than the outage itself would have caused — because now two different partial-looking answers have appeared with no explanation of either.
Interruption point 3: near-complete, resume don't restart
Failing when a response is 90% done calls for resume, not restart — the cost of throwing away a nearly-finished, probably-correct answer is disproportionate to the sliver of missing content, and a full regeneration risks producing a different (and possibly contradictory) ending to a passage the user already trusts.
The design principle: the closer to completion a stream dies, the more the interface should treat the existing text as provisionally final and offer a targeted patch rather than a wholesale redo.
Practical approach for a near-complete break:
- Detect proximity to completion where feasible — e.g., the model had already emitted a natural closing structure (a closing bracket, a final list item, a summary sentence starter) before cutting off.
- Offer
Finish this responseas the primary action, scoped to complete only the trailing gap, over a fullRegeneratebutton placed as secondary. - If your model/API doesn't support true resume (many streaming APIs don't preserve server-side generation state across a dropped connection), be honest about it — label the action "Regenerate ending" rather than implying a seamless splice.
- Timestamp or otherwise flag the moment of interruption even after a successful resume, so a user reviewing the exchange later understands part of the answer arrived in two passes.
| Interruption point | What's already true on screen | Right recovery | Wrong recovery |
|---|---|---|---|
| Before first token | Nothing rendered | Clean retry | Indefinite spinner |
| Mid-stream | Partial, unfinished sentence | Preserve + mark + continue/retry | Silent freeze or full restart with no marker |
| Near-complete | Nearly whole answer | Resume/patch the gap | Full regeneration discarding good text |
Copy and signal design for the break itself
The words and icons you use at the moment of interruption matter as much as the state machine, because this is the one place users get direct, honest information instead of inferring it from silence. The direct answer: use plain, non-alarming language that names the mechanical failure (connection, timeout, provider error) without overstating risk, and pair it with one clear next action.
A few copy patterns that hold up across contexts:
- Name the failure type simply: "Connection interrupted" reads better than a raw error code, and better than a vague "Something went wrong" that gives the user no sense of whether retrying will help.
- Avoid false urgency. A stalled stream is rarely a safety event; copy that reads as alarmed ("Critical error!") for a routine timeout trains users to distrust your alerts generally.
- Keep the action verb concrete. "Retry," "Continue," "Regenerate ending" each imply a different mechanism — don't collapse them all into one generic "Try again" if your system actually behaves differently underneath.
- Log the interruption in conversation history, not just in the live UI — if a user returns to a saved thread later, the marker should still be there, not silently smoothed over.
Building this vocabulary once, rather than ad hoc per screen, is the same discipline the complete guide to UX of failure argues for more broadly: failure states are a design surface with reusable primitives, not a one-off you write when a bug ticket forces it. It's also worth treating stream interruption copy as a sibling problem to microcopy for hallucinated answers — both are moments where the interface has to tell the truth about an AI system's limits without undermining the product's usefulness.
Where Prodinja fits
Prodinja's UX of Failure module ships a Timeout pattern and a Low-confidence pattern, both designed as reusable copy and interaction building blocks rather than one-off error strings. If you're speccing the exact moment a stream stalls or cuts out, those patterns give you a starting structure — wording, placement, and the retry/continue affordance pairing — that you adapt to your own streaming architecture rather than write from scratch. It's a prototype experience for exploring failure-state design, not a live monitor of your production stream.
Teams mapping this against real usage often start from the Customer Journey emotion curve to locate exactly where in a session stream interruptions do the most trust damage — see the complete guide to customer journey mapping for how that curve gets built, and the complete guide to jobs-to-be-done for reasoning about what job the user was trying to finish when the stream broke.
Key Takeaways
- Streaming failure needs three distinct recovery states, not one generic error screen: before-first-token (clean retry), mid-stream (preserve and mark), near-complete (resume or patch).
- Partial text reads as complete by default — a truncated sentence with no visual marker is functionally indistinguishable from a finished one, which is the core anti-pattern to eliminate.
- Never silently freeze or discard rendered tokens — a state machine with explicit
complete/interrupted/erroredstates should drive what the UI shows, not the last frame that happened to render. - Match the recovery cost to how much already rendered — retrying from zero is cheap early in a stream and wasteful late in one; resume-oriented recovery protects the user's already-invested reading time.
- Copy should name the mechanical failure plainly without false urgency, and the interruption marker should persist in saved conversation history, not just the live view.
- Prodinja's Timeout and Low-confidence patterns are useful starting copy and interaction structures for this exact moment, not a working monitor of your own stream.
Frequently Asked Questions
What should happen when a streaming AI response cuts off mid-sentence?
Keep the partial text visible, insert a clear visual marker at the exact cutoff point, and offer a Continue or Retry action — never leave the sentence looking finished or silently freeze the display with no explanation.
How do you design retry behavior differently for early vs. late stream failures?
Early failures (before or just after the first token) warrant a clean full retry since little or no content is lost. Late failures (near completion) warrant a resume or "finish the ending" action so you don't discard an answer the user has mostly already read and trusted.
Is it bad UX to show a generic error message for every streaming failure?
Yes — a single generic message collapses three functionally different situations (never started, broke partway, nearly finished) into one, which prevents the user from understanding what recovery action actually makes sense for what happened.
Should a partial AI response ever be treated as final?
No — a partial response should always carry an explicit interruption marker, even after the user dismisses the error, including in saved conversation history, so nobody later mistakes an incomplete answer for a complete one.
Can streaming APIs resume generation after a dropped connection?
It depends on the provider and architecture — some support true server-side resume from a cutoff point, while many require a fresh generation call; if your stack can't truly resume, label the recovery action honestly (e.g., "Regenerate ending") rather than implying a seamless splice.