An API is a good product when developers reach working code fast, keep expanding how much of it they use, and rarely hit friction your infrastructure dashboards can't see. The signal isn't uptime or requests-per-second — those describe your servers. Track acquisition, activation (especially time-to-first-call), engagement breadth, error quality, and retention instead.
Quick Answer: Uptime and requests-per-second tell you your servers are alive, not that your API is a good product. Track time-to-first-call, endpoint breadth per key,
4xx/5xxquality, and 30/60/90-day retention instead — and treat raw call volume as a vanity metric until you normalize it.
Why Uptime and RPS Describe Your Servers, Not Your Product
Uptime, requests-per-second, and p99 latency answer "is the system running," not "are developers succeeding with it." A perfectly available API that developers abandon after one failed integration attempt is an infrastructure success and a product failure. The two questions need entirely different instrumentation, and most API teams only build the first kind.
This isn't a knock on infrastructure monitoring — it's foundational, and Google's Site Reliability Engineering discipline formalized it well with the four golden signals: latency, traffic, errors, and saturation. Those signals were designed to answer "will this system stay up," which is a real and necessary question. They were never designed to answer "does a developer succeed after they hit 'Create API Key.'"
Conflating the two is how a team ships a green status page for months while developer activation quietly collapses. Marty Cagan's writing at the Silicon Valley Product Group has long drawn this same line for product teams generally: an actionable metric changes what you build next; a vanity metric just moves and makes everyone feel busy. Uptime moves. It rarely tells you what to build.
A few questions infrastructure dashboards structurally cannot answer:
- Did a new developer get from signup to a working call today, or did they give up at the auth step?
- Are developers using three endpoints or one — and is that one endpoint even the one you want them anchored to?
- Are your
4xxerrors dropping because docs got clearer, or because frustrated developers stopped trying? - Which cohort of API keys created 90 days ago is still calling you today?
Answering those requires a second, entirely separate metric stack — the subject of the rest of this piece. If you haven't yet nailed down the product fundamentals underneath these metrics — resource design, versioning, developer experience — our complete guide to API product design is a useful starting point before you start instrumenting.
The Five-Layer Metric Stack for API Products
A useful API metric stack has five layers: acquisition, activation, engagement, quality, and retention. It mirrors the classic AARRR funnel Dave McClure popularized under the name "pirate metrics," adapted for developers instead of consumers. Each layer answers a question the layer beneath it can't.
| Layer | Core Question | Primary Metrics | Where It Typically Breaks |
|---|---|---|---|
| Acquisition | Are the right developers signing up? | Signups, source/channel, key-creation rate | Wrong audience, high signup-to-key drop-off |
| Activation | Do they reach working value fast? | TTFC (time-to-first-call), time-to-first-success | Docs, auth friction, unclear errors |
| Engagement | Do they use more of what you built? | Endpoint breadth per key, calls per active key | Single-endpoint dependency, shallow integration |
| Quality | Is it reliable enough to build on? | 4xx/5xx rate by cause, p95/p99 latency | Silent failures, ambiguous error codes |
| Retention | Do they keep building on you? | 30/60/90-day active keys, churned-key reasons | Migration to a competitor, feature ceiling |
Read top to bottom, this table is also a diagnostic order. If retention is weak, check quality first, then engagement, then activation — a leak upstream shows up as a symptom two or three layers downstream, and teams frequently fix the wrong layer because that's where the pain became visible.
Acquisition: Are the Right Developers Signing Up?
Acquisition for an API product isn't just signup count — it's whether the developers signing up match the workflow you actually built for. A spike in signups from a channel that never converts to a created API key is a targeting problem, not a growth win.
Track signups by channel (docs search, partner integration, marketplace listing, direct sales) and split each channel's signup-to-key-creation rate separately. A channel that produces accounts but not keys is usually a mismatched audience — evaluators, students, or bots — rather than a prospect worth optimizing your onboarding around.
Two numbers worth separating here:
- Account-to-key conversion — did the person who signed up ever generate credentials?
- Key-to-first-call conversion — of those who got credentials, who actually tried them?
If the gap between (1) and (2) is large, the problem sits in your activation layer, covered next — not in acquisition itself.
Activation: Time-to-First-Call and the Race to First Value
Activation is the layer that determines whether every other metric ever gets a chance to matter, and its headline number is TTFC — time-to-first-call, the elapsed time between key creation and the first API request. A second, stricter number matters more: time-to-first-success, the time to a first successful (2xx) call, since a fast failed call teaches a developer nothing except that your API is confusing.
Stripe's developer-tooling team is widely credited in the API industry for treating "time to a working Hello World" as close to a north-star metric, embedding live-feeling code samples directly in docs so a developer never has to leave the page to get a first response. You don't need Stripe's scale to borrow the instinct: measure the clock from key creation to first 2xx, and treat every minute in that window as a place a developer could quit.
Framing activation around Jobs to Be Done helps clarify what "first value" even means for your API — a developer isn't hiring your endpoint, they're hiring it to finish a task. Our Jobs to Be Done guide walks through mapping that underlying job, which is exactly what should define your "first success" event, rather than an arbitrary "first 200 response" that happens to be an empty list.
Here's an illustrative activation funnel — proportions to model where a typical stack tends to leak, not benchmark figures from any real company:
| Funnel Stage | Illustrative Conversion | What Usually Blocks Developers Here |
|---|---|---|
| Account created → API key generated | ~70-80% | Confusing signup flow, no clear "get your key" prompt |
| Key generated → first call attempted | ~50-65% of those | Docs don't show a copy-pasteable, runnable example |
| First call attempted → first call succeeds | ~40-60% of those | Auth header errors, unclear required fields |
| First success → second distinct endpoint called | ~30-45% of those | No obvious "next step" after the first example |
| Any usage → sustained use past 30 days | ~15-25% of those | No clear path from sandbox to production |
Read this funnel as a map of where to instrument, not a target to hit. The step "first success → second distinct endpoint called" is worth watching closely — it's the earliest signal of the engagement layer below, and a common place where an ambiguous or under-specified API contract quietly kills momentum. If your errors here trace back to fields nobody agreed on the meaning of, the issue is upstream in how the API contract itself was specified, not in the docs describing it.
Engagement: Endpoint Breadth and Calls per Active Key
Engagement measures breadth and depth of real usage, not volume. The two numbers that matter most are endpoint breadth per key (how many distinct resources a given key touches) and calls per active key (usage normalized per developer, not summed across all of them).
A key that only ever calls one endpoint is fragile — that developer is one product decision away from building the rest of their integration somewhere else, or never expanding past a pilot. Breadth is a leading indicator of how deeply embedded you are in a developer's actual system, which is a much better retention predictor than total traffic.
This is also where your resource model shows up in the data. If breadth stalls at one or two endpoints regardless of what you build, it's worth revisiting whether your API resources are modeled as the nouns developers actually think in — a mismatched resource model quietly caps engagement no matter how much you invest in docs.
Watch for these engagement patterns:
- Wide, shallow keys touching many endpoints with only a handful of calls each — often exploratory or evaluation usage.
- Narrow, deep keys hammering one endpoint — often a working integration that never expanded, or a batch job.
- Wide, deep keys — your healthiest cohort, and the pattern worth reverse-engineering into your onboarding.
Quality: Error Rates and Latency, Read as Product Signals
Split every error rate by cause before you trust it. A 4xx rate is a product signal — it usually means your contract, docs, or validation messages confused a real developer. A 5xx rate is mostly an infrastructure signal — your system failed regardless of what the developer sent. Blending them into one "error rate" hides which team needs to act.
Latency deserves the same split. p50 latency is an infra health number; a rising p95 or p99 specifically for endpoints developers call during onboarding is a product number, because it's the number shaping a new developer's first impression, not your steady-state average.
Expected latency also depends on the API style you chose. A chatty REST resource graph and a single well-designed GraphQL query have very different latency profiles for the same underlying task, so "good" p95 numbers aren't universal — if you're revisiting that choice, our REST vs RPC vs GraphQL decision framework covers the trade-offs that latency numbers alone won't show you.
Retention: The Metric That Actually Predicts Revenue
Retention is the layer that turns everything above it into a business outcome, measured as the share of API keys still making calls at 30, 60, and 90 days after first activation. A high activation rate feeding a steep retention drop-off means you're good at first impressions and bad at sustained value — a pattern that looks healthy on a dashboard and terrible in a renewal conversation.
Cohort the curve by signup month rather than looking at one blended number, and pair it with churned-key exit reasons wherever you can capture them — migration to a competitor, hitting a feature ceiling, or the integration project simply getting deprioritized internally. Each has a different fix. Mapping this against the developer's own broader journey — evaluation, first integration, production rollout, scaling — usually shows retention risk concentrating at one specific transition, most often sandbox-to-production.
Separating Infrastructure Metrics From Product Metrics
Infrastructure metrics tell you whether your systems are healthy; product metrics tell you whether developers are succeeding because of what you built. Conflating them means a green status page can mask a broken developer experience, and a real product metrics dip can get misdiagnosed and routed to an on-call engineer instead of a PM.
| Infrastructure Metric | What It Tells You | Product Metric | What It Tells You |
|---|---|---|---|
| Uptime / availability % | Servers are reachable | Time-to-first-call (TTFC) | Developers reach value quickly |
| Requests per second | Raw system throughput | Calls per active key | Depth of real usage, normalized |
Server-side p50/p99 latency | System responsiveness | Endpoint breadth per key | Breadth of adoption across your surface |
5xx error rate | Ops/infra reliability | 4xx rate by cause | Contract clarity, doc quality |
| Deploy frequency, MTTR | Engineering operational health | 30/60/90-day active-key retention | Developers keep choosing to build on you |
Both columns are legitimate and both need owners. The mistake isn't tracking infrastructure metrics — it's stopping there and assuming a healthy left column implies a healthy right one. In practice they diverge constantly: 99.99% uptime with a 4xx rate climbing because of an undocumented breaking change is a completely normal, completely common state for an API to be in.
The Vanity Metric That Fools Almost Every API Team
Total API calls is the single most misleading number in API product management. It rewards a retry storm, a poorly-built polling loop, and one power user's overnight batch job exactly as much as genuine, broad adoption. A number that goes up regardless of whether your product is actually improving is close to the textbook definition of a vanity metric that Alistair Croll and Benjamin Yoskovitz laid out in Lean Analytics: it feels good, and it tells you nothing to do next.
Consider a concrete case: one client integration that polls a status endpoint every second contributes roughly 2.6 million calls a month, all by itself, with zero additional developers served. If that client happens to launch the same week you ship a genuinely better onboarding flow, your total-calls chart will show a "win" that has nothing to do with the work you actually shipped.
Signs a volume metric has drifted into vanity territory:
- It moves up or down without any corresponding change in active-key count.
- A small number of keys account for a disproportionate share of the total.
- Nobody on the team can say what action they'd take if the number dropped 20% tomorrow.
The fix is normalization, not abandonment: report calls per active key, segment by endpoint, and always pair a volume number with an active-key count in the same chart so a reader can't mistake one power user for a trend.
Turning Metrics Into a Roadmap
Metrics only matter once they change what you build next. Once your dashboards show exactly where developers stall — a documentation gap at first call, a missing endpoint blocking a common workflow, an error message nobody can decode — the next problem is deciding which fix earns engineering time first, and that's a prioritization problem, not a metrics problem.
A workable loop looks like this:
- Instrument the five-layer stack above so stall points are visible, not anecdotal.
- Find the sharpest drop — the funnel step or cohort with the steepest decline.
- List candidate fixes for that specific stall point, not a generic backlog grooming pass.
- Score and rank the candidates against each other, not just against gut feel.
- Fund the highest-leverage fix, then re-measure the same funnel step before moving to the next one.
Step four is where most API teams stall, because "which fix matters most" is a judgment call that's easy to get wrong under pressure from whichever team shouts loudest. This is the gap Prodinja's RICE and Kano Prioritization tools are built to close.
Once your activation funnel or endpoint-breadth data flags where developers actually stall, you can score each candidate improvement's reach, impact, confidence, and effort, and use Kano categorization to separate table-stakes fixes (developers expect them; skipping them causes churn) from genuine delighters worth a bigger engineering bet. The tools don't tell you what's broken — your metric stack does that — they help you rank what to fund once you already know.
Key Takeaways
- Uptime and RPS measure your servers, not your product — track a separate stack for developer success.
- Time-to-first-call (
TTFC) and time-to-first-success are the single highest-leverage numbers in the activation layer. - Endpoint breadth per key predicts retention better than total call volume ever will.
- Split
4xxfrom5xx— one is a product signal about your contract and docs, the other is mostly infrastructure. - 30/60/90-day active-key retention is the metric that turns everything upstream into a real business outcome.
- Total API calls is a vanity metric unless normalized per active key and segmented by endpoint.
- Metrics identify where developers stall; a scoring framework like
RICEorKanodecides what you fund next.
Frequently Asked Questions
What is a good time-to-first-call for an API?
There's no universal benchmark, because it depends on your API's complexity and auth model — but the useful target is directional, not absolute: measure your own current TTFC, then work to cut it, especially the gap between first attempted call and first successful one, which is usually where the real friction hides.
How do you measure API activation rate?
Define a "first success" event tied to the developer's actual job-to-be-done — not just any 2xx response — then measure the share of newly created keys that reach it within a set window, commonly 24 or 72 hours. Track it as a cohort rate over time, not a single snapshot number.
What's the difference between API product metrics and API performance metrics?
Performance (or infrastructure) metrics — uptime, RPS, server-side latency — describe whether your systems are healthy. Product metrics — TTFC, endpoint breadth, active-key retention — describe whether developers are succeeding because of what you built. Both need dashboards; only one tells you what to build next.
Is requests per second a good API KPI for product managers?
Not on its own. Requests per second is a legitimate infrastructure capacity metric, but as a product KPI it's easily inflated by retries, polling, and a handful of high-volume clients, which makes it a poor stand-in for adoption, satisfaction, or developer success.
How many API metrics should a product manager actually track?
Fewer than most teams start with. One metric per layer — acquisition, activation, engagement, quality, retention — plus one normalized volume number is usually enough to run a monthly review; more than that tends to produce a dashboard nobody actually reads before a roadmap decision.