Agent observability means capturing a step-by-step trace of every tool call, argument, and decision an agent makes—not just its final output—so you can see where a multi-step task went wrong. Standard LLM logging only shows a request and response; agents need trace-level visibility into loops, retries, and tool selection to be debuggable at all.
Quick Answer: Trace every step (tool chosen, arguments passed, result, latency, cost) as a linked chain, then monitor steps-per-task, tool error rate, loop rate, and cost per completion. A step-budget alert catches runaway retry loops before they drain your token spend.
A single-turn LLM call has one failure surface: the output is wrong or it isn't. An agent — an LLM that plans, calls tools, reads results, and decides its next move — multiplies that surface across every step in the chain. It can pick the wrong tool. It can pass malformed arguments. It can misread a tool's output and loop back into the same failed call. It can simply keep going past the point where it should have stopped. None of this shows up in a request/response log, which is exactly why teams that ship agentic features without step-level tracing end up debugging blind, reconstructing what happened from a support ticket and a hunch.
This piece assumes you already buy the case for LLM observability generally — if you haven't made that case internally yet, our complete guide to LLMOps observability covers the broader foundation this article builds on.
Why Agents Fail Differently Than Single-Turn LLM Calls
Agent failures concentrate in four places single-turn calls never touch: tool selection, argument construction, loop detection, and step budgeting. Each is a distinct failure mode with its own signature, and conflating them into one generic "agent broke" bucket is why so many agent postmortems go nowhere.
Tool selection errors happen when the agent picks a plausible-sounding tool that isn't actually right for the task — calling a search_orders function when the user asked about a refund_status, for instance. The call succeeds mechanically; it's just the wrong call. This is the hardest failure mode to catch with generic error monitoring, because nothing throws an exception.
Argument errors happen when the right tool gets called with malformed, hallucinated, or stale arguments — a date format the API rejects, a customer ID copied from three steps earlier that's no longer relevant, a required field the agent invents a plausible-looking placeholder for. These often do throw, which is why they're the easiest of the four to instrument first.
Loop failures happen when an agent re-attempts a failed step without changing its approach, or ping-pongs between two tools that each partially satisfy a condition the other undoes. A retry with no meaningful change in arguments or strategy is a loop, not persistence — and undetected, it's the failure mode that burns the most tokens per incident.
Step-budget failures are the catch-all: the agent hasn't technically errored or looped, it's just still going, deeper into a task than the value justifies. Anthropic's own guidance on building effective agents and OpenAI's published agent-design patterns both converge on the same fix here: bound the agent explicitly with a maximum step count or token budget rather than trusting it to self-terminate, because nothing in an autoregressive model's training pressures it to recognize "I should stop now."
| Failure mode | What it looks like in a trace | Typical root cause |
|---|---|---|
| Tool selection error | Tool call succeeds, output is irrelevant to the ask | Ambiguous tool descriptions, overlapping tool scopes |
| Argument error | Tool call throws, or returns an empty/error payload | Stale context, hallucinated fields, schema drift |
| Loop | Same tool + similar arguments repeating 2+ times | No repeat-detection, no strategy change on retry |
| Step-budget overrun | Step count climbing with no terminal action in sight | No max-step cap, vague or unbounded task framing |
Anatomy of a Step Trace
A usable agent trace records every step as a discrete, linked event: which tool was called, what arguments went in, what came back, how long it took, and what it cost — chained together so you can replay the whole task in order. Without that structure, you're stuck grep-ing raw logs after the fact.
At minimum, each step in the trace should carry:
- Step ID and parent task ID — so every step ties back to the originating user request or triggering event, not just floats as an isolated log line.
- The agent's stated reasoning or plan fragment, if your framework exposes one — even a short "why this tool" note massively shortens debugging time.
- Tool name and full argument payload — the exact call made, not a summary of it.
- Raw tool response (or a truncated version with a pointer to the full payload) — including error codes and messages, not just success/fail.
- Latency and token/cost consumption for that single step — this is what step-level cost analysis is built from.
- A step outcome flag — success, retryable error, terminal error, or loop-suspected — set by your own instrumentation, not left implicit.
Where This Trace Data Should Live
Step traces should land somewhere queryable, not just in a scrolling log tail — you want to ask "show me every task where steps exceeded 15" as a query, not a manual search. Whether that's a dedicated tracing backend, a structured events table, or an existing observability stack extended with an agent_step event type matters less than the discipline of writing the same fields every time.
This is the same instinct behind three dashboards you should build on day one for any AI product — the trace is the raw material one of those dashboards is built from. If you're only watching output quality, you're missing the layer where agent-specific failures actually originate, which is a distinct problem from the general output quality drift with no stack trace that affects single-turn generation too.
The Metrics That Actually Tell You Something
Four metrics separate a healthy agentic workflow from one quietly bleeding cost and reliability: steps per task, tool error rate, loop rate, and cost per completion. Each answers a different question, and none of them substitutes for the others.
Steps per task is your efficiency baseline. Track the distribution (median and p90, not just an average — a few pathological long-tail tasks will otherwise hide inside a reasonable-looking mean), and watch it over time as you change prompts or tool sets. A creeping p90 is often the earliest signal something upstream degraded, well before users complain.
Tool error rate is the share of tool calls that return an error or malformed result, broken down per tool. A single tool with a disproportionate error rate usually points to a schema mismatch, a rate limit being hit, or a description that's misleading the agent about required arguments — it's a targeted fix, not a whole-system one.
Loop rate is the share of tasks that include a detected repeat-without-progress pattern. This is the metric most teams skip instrumenting first and regret it, because loops are simultaneously the most expensive failure mode and the hardest to notice from output alone — the final answer can look completely fine even after the agent burned through a dozen unnecessary calls to get there.
Cost per completion ties the other three together into a number leadership actually cares about. An agent with a high loop rate but an acceptable-looking success rate is still a cost problem, and cost-per-completion is the metric that surfaces it when success-rate dashboards alone would not.
| Metric | What it catches | Healthy signal | Warning signal |
|---|---|---|---|
| Steps per task (p90) | Inefficient planning, scope creep | Stable or declining over time | Rising trend across releases |
| Tool error rate (per tool) | Argument/schema issues, tool misuse | Under a few percent, tool-specific | One tool spiking well above baseline |
| Loop rate | Retry-without-progress patterns | Near zero for well-scoped tasks | Any non-trivial, non-decreasing rate |
| Cost per completion | All of the above, in dollar terms | Predictable, bounded per task type | Wide variance or upward drift |
A Worked Example: An Agent Burning Tokens in a Retry Loop
Picture an agent tasked with updating a customer's shipping address across two backend systems. It calls update_address in System A successfully, then calls sync_address to propagate the change to System B — but System B rejects the payload because a required region_code field is missing from the copied context.
Instead of surfacing the schema mismatch, the agent reinterprets the error as "the address wasn't set" and calls update_address again, then sync_address again, hitting the identical rejection. Over 40 steps and several minutes, it repeats this pair nine times, each cycle consuming a full reasoning pass plus two tool calls, before a step-budget cap finally forces termination with no successful outcome.
In the trace, this is visible immediately: nine near-identical (update_address, sync_address) pairs with the same region_code error message, no change in arguments between attempts, and step count climbing steadily with zero net progress toward a terminal state. In a raw output log with no step trace, all a team would see is: task took unusually long, then failed — with no visibility into why, or that it was a loop rather than, say, a slow but legitimate multi-step task.
The Alert That Catches It
The fix is a repeat-call detector: an alert that fires when the same tool name plus a near-identical argument hash (ignoring volatile fields like timestamps) appears two or more times consecutively within one task, without an intervening change in strategy. Pair it with a hard step-budget ceiling per task type, tuned from your steps-per-task p90 data rather than picked arbitrarily.
- Trigger condition: same tool + argument-hash match, 2+ consecutive occurrences, same task ID.
- Secondary trigger: step count exceeds a configured ceiling with no terminal-success flag.
- Action on fire: halt the task, log the full trace for review, and — where user-facing — degrade gracefully to a human handoff rather than silently retrying forever.
- Tuning input: your own steps-per-task and loop-rate history, not a generic industry default, since acceptable step counts vary enormously by task complexity.
This is precisely the gap between watching an agent in production and having examined the workflow's failure modes before it shipped. Prodinja's Agentic-workflow critique concept is designed to walk you through where a multi-step flow can go wrong — which steps are load-bearing, where a tool's error handling is underspecified, where a loop could form — before you build it, as a structured pre-mortem exercise inside the prototype. It's a natural complement to the traces described above: one is the pre-build map of where things could fail, the other is what actually happened once real traffic hit the flow. Neither replaces the other.
Turning Traces Into a Feedback Loop, Not Just a Postmortem Tool
A step trace that only gets opened after an incident is doing half its job. The other half is feeding failures back into your eval suite so the same loop, tool-selection error, or argument mismatch gets caught before it reaches production again. Our piece on closing the loop from production failures to eval cases covers the mechanics of that pipeline in depth — traces are the input, regression eval cases are the output.
In practice, this means every incident your loop-rate alert or tool-error-rate spike surfaces should end with a question: does this specific failure now have a corresponding eval case that would catch it on the next model or prompt change? If the answer is no, the trace stayed a one-off curiosity instead of becoming durable protection. Teams that skip this step tend to rediscover the same handful of failure patterns every few months, each time as if for the first time.
Where Traces Fit Alongside Product Discovery
None of this instrumentation replaces upfront thinking about what the agent should actually be doing. A trace tells you how a workflow failed; it doesn't tell you whether the workflow was solving the right job in the first place. That's a separate, earlier-stage question — one closer to Jobs to Be Done framing or mapping the surrounding customer journey than to observability tooling. Get the job wrong and even a perfectly-traced, zero-loop agent is just efficiently doing the wrong thing.
Key Takeaways
- Agents fail across four distinct surfaces — tool selection, argument construction, loop formation, and step-budget overrun — and each needs its own signal, not one generic "agent broke" alert.
- A usable step trace links every tool call, argument payload, result, latency, and cost back to a parent task ID, so a failure can be replayed in order rather than reconstructed from guesswork.
- Steps per task, tool error rate, loop rate, and cost per completion are the four metrics worth dashboarding first; track distributions (p90), not just averages, since long-tail tasks hide inside means.
- Loop rate is the metric most teams instrument last and need most — it's simultaneously the biggest silent cost driver and invisible from output quality alone.
- A repeat-call detector plus a tuned step-budget ceiling is the concrete alert that catches a retry loop before it exhausts a token budget, not after.
- Traces should feed back into your eval suite, not just close out an incident — an unturned trace is a failure pattern waiting to recur.
- Pre-build critique and post-deploy tracing are complementary, not redundant — one maps where a flow could break, the other shows where it actually did.
Frequently Asked Questions
What is agent observability?
Agent observability is the practice of capturing step-level traces of an AI agent's tool calls, arguments, and decisions — not just its final output — so multi-step failures like loops, wrong tool selection, and argument errors can be diagnosed. It extends standard LLM logging with a per-step, linked-chain structure.
How do you detect an infinite loop in an AI agent?
Detect loops by hashing each tool call's name plus its arguments (excluding volatile fields like timestamps) and flagging when the same hash appears two or more times consecutively within one task without a strategy change. Pair this with a hard step-budget ceiling as a backstop for loops the hash check misses.
What metrics should I track for agentic workflows in production?
Track steps per task (median and p90), tool error rate per tool, loop rate, and cost per completion. These four cover efficiency, tool reliability, runaway-retry risk, and the dollar impact of all three combined — no single metric substitutes for the others.
Why don't normal LLM logs catch agent failures?
Standard request/response logs only capture the final output of a task, not the intermediate tool calls and decisions that produced it. An agent can loop through a dozen failed steps and still emit a plausible-looking final answer, so output-only logging misses the cost and reliability problem entirely.
How is agent tracing different from evaluating single-turn LLM outputs?
Single-turn evaluation checks whether one input produces a correct output. Agent tracing follows a chain of multiple tool calls and decisions within one task, because the failure can occur at any step along that chain — not just in the final response the user sees.