A feature's gross margin at 100 users tells you almost nothing about its margin at 100,000, because AI cost scales with usage intensity, not seat count. Protect margin by modeling per-request cost against a realistic usage distribution before launch, then defending it with caps, tiering, caching, and pricing that passes marginal cost through to heavy users.
Quick Answer: LLM features have marginal cost per request, unlike most SaaS features. Model blended gross margin across a free/paid cohort mix, identify where power users blow past your assumed cost envelope, and build in at least one lever — usage caps, model tiering, semantic caching, or usage-based pricing — before you scale, not after finance flags it.
Why AI Features Break the Traditional SaaS Margin Model
Traditional SaaS gross margin is dominated by fixed infrastructure cost divided across a growing user base, so margin improves with scale. AI features invert this: every request incurs a real, variable compute cost, so margin can compress as usage grows instead of expanding.
A conventional software feature — a dashboard, a report export, a search filter — costs close to zero marginal dollars per use once it's built. Serving the 10,000th user costs almost the same as serving the first. That's the economic logic behind SaaS multiples: revenue scales faster than cost, so gross margin climbs toward 80-90% as the business grows, a pattern well documented in software cost-structure research from firms like Bessemer Venture Partners' State of the Cloud reports.
An LLM-backed feature doesn't behave this way. Every inference call — a chat response, a document summary, an agentic tool loop — consumes tokens, and tokens have a real, metered dollar cost from the model provider. Add retrieval calls, embeddings, and any second-pass verification step, and the cost compounds per request, not per user. A feature that costs $0.02 to delight your first 100 users can cost $200,000/month once 100,000 users hit it at realistic frequency.
The Core Mechanism: Cost Follows Usage Intensity, Not Headcount
The critical variable most teams underestimate is usage intensity per user, not user count. Two customers on the same plan can generate wildly different AI COGS depending on how long their sessions run, how much context they attach, and whether they loop the model repeatedly.
This is where unit economics modeling has to start:
- Cost per request — average tokens in, tokens out, and the per-token rate of whichever model tier serves that request.
- Requests per active user per period — not an average across all users, but a distribution, since usage is almost always long-tailed.
- Active user growth — how many users hit the feature at all, separate from how hard each one uses it.
Multiply naively using only averages and you'll underestimate cost, because the tail — not the median — is what erodes margin.
Building the Per-Request Cost Model
A defensible AI unit economics model starts with a single number: fully-loaded cost per request, built from every compute step a request actually triggers, not just the headline model call. Skipping hidden steps — retrieval, re-ranking, a verification pass — is the most common way these models understate cost.
Break the cost of one request into its components:
- Input tokens: the prompt, plus any system instructions, retrieved context, or conversation history sent with every call.
- Output tokens: the generated response — often the larger cost driver for long-form generation or agentic reasoning traces.
- Auxiliary calls: embeddings for retrieval, a smaller classifier model, a moderation pass, or a self-critique loop before the final answer ships.
- Infrastructure overhead: vector database queries, orchestration compute, logging — smaller per request, but non-zero at scale.
| Cost component | Typical driver | Scales with |
|---|---|---|
| Input tokens | Prompt + retrieved context + history | Conversation length, RAG context size |
| Output tokens | Generated response length | Task complexity, verbosity settings |
| Auxiliary model calls | Embeddings, routing, verification | Architecture choices, not user behavior |
| Infra/orchestration | Vector search, logging, retries | Request volume, error rate |
The table above shows why architecture decisions, not just the headline model, drive cost. A feature that adds a verification loop to improve accuracy has just doubled or tripled its per-request cost — a trade-off worth making deliberately, using something like a formal trade-off analysis rather than discovering it in a cloud bill.
Blended Gross Margin: Combining Cost With Your Pricing Model
Blended gross margin is revenue minus AI COGS minus supporting infrastructure, divided by revenue, calculated across your actual mix of users rather than a single "typical" one. It's the number finance actually cares about, and it can look healthy in aggregate while individual segments are already underwater.
The formula is simple; the inputs are where teams go wrong:
Blended gross margin = (Revenue - AI COGS - supporting infra cost) / Revenue
Revenue per user is usually a known, fixed number by plan. AI COGS per user is not fixed — it's a function of how much that specific user invokes the feature. A single blended margin number, calculated across your whole base, can hide the fact that your 90th-percentile users are already unprofitable while your median user still generates healthy contribution margin.
Cohort Example: Free vs. Paid Usage Erodes Margin Differently
A simple two-cohort model — free-tier users and paid-tier users — already shows how margin diverges once usage intensity is added, well before any single user is individually unprofitable in isolation. This is the model every AI product leader should build before a feature ships broadly, not after.
Assume a feature costs $0.03 per request fully loaded (input, output, and one retrieval call), and a paid plan charges $20/month with an assumed average of 150 requests per user per month.
| Cohort | Plan price | Assumed requests/mo | AI COGS/mo | Gross margin |
|---|---|---|---|---|
| Free tier | $0 | 40 (capped) | $1.20 | N/A (loss leader) |
| Paid, typical | $20 | 150 | $4.50 | 77.5% |
| Paid, power user | $20 | 800 | $24.00 | -20% |
| Paid, agentic power user | $20 | 800 (with tool loops, ~3x tokens/req) | $72.00 | -260% |
The typical paid user looks great: 77.5% gross margin, comfortably within SaaS norms. But the power-user row shows the model breaking — a user well within "normal engaged use" for many products is already margin-negative, and an agentic user chaining multiple tool calls per request is catastrophically so. Blended margin across the whole cohort can still look acceptable if power users are a small percentage of the base — until that percentage grows, which engaged features are specifically designed to encourage.
This is the trap: the same product behaviors you optimize for in growth and engagement metrics — session frequency, feature stickiness, deeper workflows — are the exact behaviors that compress AI gross margin. Growth and margin can pull in opposite directions on an unmetered AI feature in a way they rarely do on a conventional software feature.
Levers to Defend Margin as Usage Scales
There is no single fix for AI margin erosion — durable protection combines usage limits, cost-aware model selection, caching, and pricing design, each addressing a different part of the cost curve. Treat them as a portfolio, not a single decision.
1. Usage Caps and Tiering
The blunt instrument, and often the fastest to ship. Cap requests, tokens, or context size per plan tier, and make the cap visible to users so it reads as a feature boundary rather than a punitive throttle.
- Set caps using your actual cost model, not a round number — a cap at the 80th-percentile usage level protects margin on the users driving the worst outcomes while leaving typical usage untouched.
- Offer an explicit overage or higher tier for power users, converting a margin problem into a monetization opportunity instead of a silent loss.
- Revisit caps quarterly — model pricing and usage patterns both shift faster than annual planning cycles.
2. Model Routing: Match Model Cost to Task Complexity
Not every request needs the most capable, most expensive model. Routing simpler requests to smaller, cheaper models and reserving frontier models for genuinely hard tasks can cut blended cost significantly without a perceptible quality drop for most requests — a practice covered in depth in guidance on routing between cheap, fast, and default models.
This requires classifying request difficulty upfront, which is itself a design decision worth making deliberately rather than defaulting every call to your most expensive model out of convenience.
3. Caching: Avoid Paying for the Same Compute Twice
Semantic and prompt caching reduce cost by recognizing when a request — or a large shared portion of one, like a system prompt or repeated context block — has effectively been seen before, and reusing the result or the cached computation instead of recomputing it. Providers increasingly discount cached input tokens substantially versus fresh ones.
Two related techniques matter here for different reasons:
- Semantic caching of LLM responses matches new queries to previously-answered similar ones, avoiding redundant generation for common questions.
- Prompt caching mechanics reduce the cost of repeated large context blocks — system instructions, long documents, tool definitions — sent on every call within a session.
Both are architecture decisions, and both directly move the "input tokens" and "auxiliary calls" rows in the cost table above.
4. Pricing That Passes Marginal Cost Through
Usage-based or hybrid pricing — a base subscription plus metered overage — aligns what a customer pays with what they cost to serve, which flat-rate pricing structurally cannot do once usage varies enough between customers. This is the most durable long-run fix, though the hardest to retrofit onto an existing flat-rate base.
Pricing researchers, including those at OpenView Partners in their annual SaaS benchmarks work, have tracked a clear multi-year shift toward usage-based and hybrid pricing specifically as AI features have entered mainstream products — a direct response to exactly this margin dynamic.
Hybrid models — a flat fee covering typical usage, metered pricing beyond it — tend to land best with customers, since pure metering makes cost unpredictable and erodes the "unlimited" expectation SaaS buyers have built up over a decade.
Making the Cost Axis Concrete Before You Ship
Margin conversations too often start as a guess made in a launch-planning meeting and get revisited only once a finance review flags an unexpected cloud bill. The fix is starting the conversation from architecture, not intuition — knowing which design choices (context size, tool-call depth, model tier, caching strategy) drive cost before a single line of the feature is built.
Key Takeaways
- AI features carry real marginal cost per request, unlike most SaaS features, so gross margin can compress as usage grows instead of improving with scale.
- Model cost per request from every compute step — input tokens, output tokens, auxiliary calls like retrieval or verification, and infrastructure overhead — not just the headline model call.
- Blended margin can mask segment-level losses: a healthy aggregate number can coexist with power-user cohorts that are already deeply unprofitable.
- The behaviors that drive engagement — frequent, deep, agentic usage — are the same behaviors that erode AI margin, creating tension between growth and profitability metrics.
- Defend margin with a portfolio of levers: usage caps and tiering, cost-aware model routing, semantic and prompt caching, and usage-based pricing that passes marginal cost through.
- Model the margin curve before launch, using a realistic usage distribution and cohort breakdown, not a single average user, so surprises surface in planning rather than in a finance review.
Frequently Asked Questions
What is unit economics for an AI feature?
AI feature unit economics is the relationship between revenue per user and the variable compute cost — tokens, model calls, retrieval, infrastructure — required to serve that user's actual usage. Unlike most SaaS unit economics, the cost side scales with usage intensity per user, not just with total headcount.
Why does gross margin drop as an AI feature scales, when SaaS margin usually improves?
Because AI compute cost is marginal and per-request, while most legacy SaaS infrastructure cost is largely fixed and amortizes across a growing user base. As an AI feature's most engaged users grow as a share of the base, their higher per-request cost pulls blended gross margin down rather than up.
How do you calculate AI gross margin (COGS) per feature?
Sum fully-loaded cost per request — input tokens, output tokens, auxiliary model calls, and infrastructure overhead — multiplied by requests per user across a realistic usage distribution, then subtract that from revenue per user and divide by revenue. Calculating from an average user alone will understate the true cost curve.
What's the fastest lever to protect margin on an existing AI feature?
Usage caps or tiering are typically the fastest to implement, since they don't require re-architecting the underlying model pipeline. Caching and model routing usually deliver larger sustained savings but take longer to build; usage-based pricing is the most durable fix but the hardest to retrofit onto an existing flat-rate customer base.
Should free-tier AI features be capped differently than paid tiers?
Yes — free tiers should be capped tightly enough that AI COGS stays a bounded, predictable acquisition cost rather than an open-ended loss, since free users generate no revenue to offset usage. Paid-tier caps should be set from your actual cost model, typically near a percentile that isolates disproportionately expensive power users without limiting typical usage.