Rate Limits Are a Product Decision

Rate limits aren't a backend safety valve bolted on after launch — they're a pricing lever, a reliability contract, and a behavior nudge in one. Design them by picking an algorithm (token bucket, fixed or sliding window), a scope (per-key or per-endpoint), and a communication contract (headers plus Retry-After) before you throttle a single real request.

Quick answer: Use token bucket for public APIs with bursty legitimate traffic, fixed window only for simple low-risk cases, and sliding window when boundary fairness matters more than memory cost. Scope limits per API key for billing and per endpoint for backend protection, expose RateLimit-* headers plus Retry-After on every response, and let your tiers — not your infrastructure team's mood — decide the numbers.

Why Rate Limits Are a Product Decision, Not an Infrastructure Afterthought

Rate limits function as a throttle on both cost and experience: they protect infrastructure from abuse, create a natural upsell path between plan tiers, and signal which client behaviors you want to encourage. Treating them as a pure ops concern — set once, tune only after an incident — throws away a lever that shapes revenue and trust at the same time.

Every limit you set answers a question your pricing page and your on-call rotation both care about: how much of this system does one customer get to consume? That single number does several jobs at once:

Protects shared infrastructure from a single noisy tenant or runaway script degrading service for everyone else.
Creates a legitimate reason for tiers — a free plan that's genuinely useful but bounded, a paid plan sized for production load.
Shapes integration patterns, nudging developers toward batching, caching, and webhooks instead of aggressive polling.
Acts as an early-warning signal, since a sudden spike in 429s often reveals abuse, a bug in a client's retry loop, or a tier that's mispriced.

A rate limit is the one place your pricing page and your on-call rotation agree on the same number.

This is one lever inside a larger discipline. If you haven't yet mapped where quotas fit alongside versioning, authentication, and error design, the complete guide to API product design is the broader frame this article sits inside.

Token Bucket, Fixed Window, or Sliding Window: Choosing Your Algorithm

Token bucket is the safest default for most public APIs, because it absorbs legitimate bursts while still enforcing a steady average rate over time. Fixed window counters are the simplest to build but let traffic double at window boundaries; sliding window variants fix that at a higher memory or compute cost. Pick based on how bursty your real traffic is, not on what ships fastest.

Each algorithm makes a different tradeoff between fairness, implementation cost, and memory footprint. None is universally correct — a batch-import endpoint and a real-time search endpoint often deserve different choices even within the same API.

Algorithm	How it works	Burst handling	Memory cost	Best fit
Token bucket	Tokens refill at a fixed rate into a bucket; each request spends one token	Allows bursts up to bucket size, then throttles to the refill rate	Low — one counter and timestamp per key	Public APIs with legitimately bursty clients
Fixed window	Counts requests inside discrete blocks (e.g., per minute) and resets at the boundary	Vulnerable to a near-2x burst spanning two windows	Very low — one counter per key per window	Simple internal services, low abuse risk
Sliding window log	Stores the timestamp of every request; counts those inside a trailing window	Smooths the boundary problem precisely	High — stores every timestamp	Low-volume APIs needing exact enforcement
Sliding window counter	Weights the previous and current window counts proportionally	Approximates the log's fairness cheaply	Low-to-moderate	High-volume, edge-level limiting

Nginx's limit_req module, documented by Nginx Inc., implements a variant of the leaky bucket algorithm — a close cousin of token bucket that smooths outbound request rate rather than just capping a count. It's worth knowing as a fifth option when you need queuing behavior instead of outright rejection.

Estimate your legitimate burst shape first — a mobile app syncing on reconnect looks nothing like a server-to-server integration polling on a timer.
Default to token bucket unless you have a specific reason (extreme simplicity needs, or precise fairness needs) to choose otherwise.
Reserve sliding-window-log precision for low-volume, high-stakes endpoints (billing, account deletion) where exactness matters more than memory.

Per-Key vs. Per-Endpoint Limits: Scoping Quotas to Match Risk

Per-key limits govern a customer's total consumption and map directly to billing tiers. Per-endpoint limits protect specific expensive operations — search, bulk export, webhook redelivery — regardless of which customer calls them. Most production APIs need both layers simultaneously, because a fair total quota can still let one costly endpoint monopolize a shared database.

Some endpoints deserve their own ceiling even inside a generous per-key allowance:

Full-text or fuzzy search, which is disproportionately expensive per request compared to a simple lookup by ID.
Bulk export or import, which can hold a connection or a worker for far longer than a typical call.
Webhook re-delivery, which a misbehaving integration can trigger in a tight loop if left unbounded.
Any endpoint proxying a third-party service, where your own rate limit protects you from a vendor's rate limit.

Because these limits key off the same nouns you modeled when designing the API's resources, getting your resource modeling nouns right upfront makes scoping quotas by endpoint far more tractable — a /search resource and a /records/{id} resource simply warrant different ceilings.

Your API's shape also decides how "per-endpoint" even applies. A single GraphQL endpoint makes route-based counting close to meaningless, since one query can request a shallow field or a deeply nested, expensive tree — a tradeoff worth weighing before you commit to a style, covered in this REST, RPC, and GraphQL decision framework.

The Standard Headers That Make Limits Predictable

Expose machine-readable limit state on every response, not just the 429 — using either the legacy X-RateLimit-Limit / X-RateLimit-Remaining / X-RateLimit-Reset convention popularized by GitHub and Twitter, or the newer IETF-standardized RateLimit-Limit / RateLimit-Remaining / RateLimit-Reset fields. Either way, headers turn throttling from a surprise into a signal clients can poll and react to before they get blocked.

Header	Purpose	Example
`RateLimit-Limit` (or `X-RateLimit-Limit`)	Maximum requests allowed in the current window	`100`
`RateLimit-Remaining` (or `X-RateLimit-Remaining`)	Requests left before throttling kicks in	`42`
`RateLimit-Reset` (or `X-RateLimit-Reset`)	Seconds or timestamp until the window resets	`30`
`Retry-After`	Sent on `429` or `503`; tells the client how long to wait before retrying	`Retry-After: 60`

The IETF's draft-ietf-httpapi-ratelimit-headers specification, developed under the HTTP working group, exists specifically because GitHub, Vimeo, Twitter, and others each shipped slightly different header names before any standard did — a fragmentation any new public API can now sidestep by adopting the standardized field names directly.

The 429 Too Many Requests status code itself traces back to RFC 6585, published by the IETF in 2012 to give rate limiting a dedicated response code distinct from a generic 403 or 503. Using it correctly, rather than reusing an adjacent code, is what lets client libraries branch on rate-limit handling automatically.

Headers are part of the contract, not an implementation detail. The same discipline that goes into specifying a complete API contract for request and response bodies should extend to documenting exactly which rate-limit fields your API guarantees and in what format.

Retry-After and the Art of a Graceful 429

A well-designed 429 response tells a client what happened, how long to wait, and what changed — not just that it failed. Pair the status code with a Retry-After header (seconds or an HTTP-date), a JSON body naming which specific limit was hit, and behavior consistent enough that retry logic can be written once and trusted everywhere.

Always return Retry-After, even as an estimate — silence forces every client to guess and invent its own backoff heuristic.
Distinguish limit types in the response body (per-minute, per-day, concurrent-connections) so SDKs and error handlers can react differently to each.
Log and expose your own 429 rates internally — a sudden spike is often a signal that a tier, a doc, or a client's retry loop is mismatched with real usage.
Recommend exponential backoff with jitter in your docs and official SDKs, so integrators aren't fighting synchronized retry storms against your own API.

A rate limit hit mid-integration is a classic low point in a developer's experience of your product. Mapping that moment explicitly — the same way you'd chart an emotion curve across any customer journey — is what separates an API that frustrates a developer from one that quietly teaches them how to use it well.

Mapping Quotas to Pricing Tiers and Documenting the Contract

Tiers should map to limits that reflect real usage patterns and willingness to pay: a free tier low enough to prevent substitution for a paid plan but usable for genuine evaluation, a mid tier sized for one production integration, an enterprise tier negotiable with an attached SLA. The limits themselves become part of the sales conversation, not just a technical config value.

The following is a starting framework to adapt, not a fixed formula — your actual numbers depend on your infrastructure cost curve and what a "typical" integration looks like for your product:

Tier	Typical rate limit	Burst allowance	What it's really selling
Free / trial	Low (tens to low-hundreds of requests/min)	Minimal	Evaluation access, abuse prevention
Starter / pro	Moderate (hundreds to low-thousands/min)	Moderate token-bucket burst	One production integration
Business	High, often per-seat or per-workspace	Generous	Multiple integrations, internal tooling
Enterprise	Custom, negotiated	Custom	SLA, dedicated capacity, guaranteed support response

Real APIs make these tradeoffs publicly. GitHub's REST API documentation publishes a limit around 5,000 requests per hour for authenticated personal-access-token users — a ceiling chosen to comfortably cover normal tooling while still bounding abuse. Stripe's API documentation similarly describes a token-bucket-style limiter around 100 requests per second in live mode, with higher ceilings negotiated case-by-case for high-volume merchants. Both are directional examples of the same principle: the number is a business decision expressed in code.

Sizing a tier's limit well starts with understanding the job the customer is hiring your API to do. A nightly batch sync and a real-time checkout flow have wildly different throughput needs — exactly the kind of distinction a jobs-to-be-done analysis surfaces before you commit to a single number per tier.

Documenting the throttling contract so it's not discovered in production

None of this matters if the limits live only in a support engineer's head or a Slack thread from launch week. The throttling contract — which headers you emit, what a 429 body contains, which tier maps to which ceiling — belongs in the same spec as your endpoints and schemas, versioned and reviewed the same way.

Prodinja's API Designing tool lets you document rate-limit headers and 429 responses alongside your endpoint definitions, so the throttling contract is visible in the spec itself rather than discovered by an integrator hitting a wall in production. It's designed as part of walking through endpoints to a runnable curl/spec artifact — the quota policy sits next to the request and response shapes it constrains, not in a separate document nobody opens.

Key Takeaways

Rate limits are policy, not just protection — they encode pricing, fairness, and trust decisions in a single number.
Token bucket is the safest default for public APIs with bursty legitimate traffic; reserve fixed window for simple, low-risk internal cases.
Layer per-key and per-endpoint limits together — a fair total quota doesn't stop one expensive endpoint from monopolizing shared infrastructure.
Expose RateLimit-* headers plus Retry-After on every response, not just the 429, so throttling becomes information instead of a surprise.
Standardize on RFC 6585's 429 status and, where possible, the IETF's draft rate-limit header fields rather than inventing your own conventions.
Map tiers to real usage patterns, not round numbers — size the free tier for evaluation and the paid tier for one production integration.
Document the throttling contract in the spec itself, so integrators discover limits by reading docs rather than by tripping a 429 in production.

Frequently Asked Questions

What's the difference between rate limiting and throttling?

Rate limiting rejects requests outright once a quota is exceeded, typically with a 429 response, while throttling delays or queues requests to smooth the rate without an outright rejection. Many production systems use both: throttling absorbs normal spikes gracefully, and a hard rate limit acts as the backstop against abuse or a runaway client.

Should I rate limit by API key or by IP address?

Rate limit by API key whenever authentication is available, since IP-based limits break down for users behind shared NAT or corporate proxies and are trivial to evade by rotating addresses. Reserve IP-based limiting for unauthenticated endpoints, like login or signup forms, where no key exists yet to key off of.

What HTTP status code should a rate-limited request return?

Return 429 Too Many Requests, the status code RFC 6585 defined by the IETF specifically for this case, rather than reusing a generic 403 Forbidden or 503 Service Unavailable. Always pair it with a Retry-After header so the client knows when to retry instead of guessing or hammering the endpoint again immediately.

How strict should a free-tier rate limit be?

Set the free tier low enough that it can't substitute for a paid plan in production, but high enough for a developer to fully evaluate the API in a single sitting — commonly tens to a few hundred requests per minute. Watch actual free-tier usage after launch and adjust; the right number is rarely obvious before real traffic arrives.

Do GraphQL APIs need rate limiting differently than REST APIs?

Yes — a single GraphQL endpoint makes simple per-route request counting close to meaningless, since one query can request a shallow field or a deeply nested, expensive tree of relations. Most teams instead score query complexity or depth and rate-limit on that computed cost, rather than on raw request count alone.