The Unit Economics of an AI Feature: Modeling Cost Per Inference Before Launch

Cost per inference is the marginal compute cost of one AI call — input tokens plus output tokens, priced at the model's metered rate — multiplied by how often an active user triggers it. Multiply that by your user base and you get a real, growing cost line that traditional software never carried, because software's marginal cost per user was effectively zero.

Quick Answer: Cost per inference = (input tokens + output tokens) × price per token × calls per active user × active users. Model this before launch — the number that's a rounding error at 1,000 users can be a six-figure monthly loss at 1,000,000.

Why AI Features Break the Zero-Marginal-Cost Rule of Software

Traditional software has near-zero marginal cost per user: serving the ten-millionth page view or database row costs fractions of a cent, so a feature that's profitable at 1,000 users is almost always more profitable at 1,000,000. AI features invert that assumption. Every call to a language model consumes metered tokens or GPU time, so cost scales with usage, not just with headcount, and it never approaches zero.

This is the single biggest mental shift required of a PM scoping an AI feature. Engineering effort used to be the main cost of a feature — a fixed, one-time investment amortized across every future user for free. With AI, you've added a recurring, usage-linked cost that behaves more like cloud hosting for a video-heavy app than like a typical CRUD feature, except the per-unit price is often higher and less predictable.

The venture research community has been documenting this gap for two years. Andreessen Horowitz's analysis of AI application economics has repeatedly flagged that AI-native software companies often run structurally lower gross margins than traditional SaaS — commonly in the 50–60% range rather than the 75–90% typical of subscription software — precisely because inference is a real cost of goods sold, not a sunk engineering cost.

Sequoia Capital's widely circulated "AI's $600B Question" analysis made a parallel point at the infrastructure level: the industry's compute spend has been running well ahead of the revenue it's currently generating. That's a macro version of the exact problem a single feature can have in miniature — the bill arrives whether or not the growth curve has caught up to it yet.

None of this means AI features are unaffordable. It means cost per inference has to become a first-class feasibility input, checked before a launch date is fixed — alongside the questions covered in a full AI feasibility pass — rather than discovered on the first month's cloud invoice.

Building the Cost-Per-Inference Model: Tokens, Calls, and Users

A cost-per-inference model has exactly three moving parts: tokens per call, calls per active user, and price per token — multiply those together and then by your user count, and you have a monthly cost estimate accurate enough to make a launch decision. Getting each input roughly right matters more than precision on any single one.

Step-by-step: assembling your own estimate

1. Count tokens per call, not words. A rough rule of thumb is 1 token ≈ 0.75 English words; measure your actual prompt and expected output length rather than guessing.
2. Separate input and output pricing. Providers price output tokens at a multiple of input tokens — often 4–5x — so a chatty response costs more than a long prompt with a terse answer.
3. Estimate calls per active user per month, not per account. A feature used once at onboarding and never again has a completely different cost profile than one used daily.
4. Multiply cost-per-call by calls-per-user to get cost per active user per month — the number that should sit next to your revenue-per-user in the same spreadsheet.
5. Multiply by projected active users at scale, not just at launch — this is where most underestimates happen, and it's the subject of the next section.

How often a user actually needs the underlying job done is not a guess you should make from a whiteboard — it's exactly what a rigorous Jobs to Be Done analysis is built to surface, including how frequently the job recurs and under what triggering conditions. A feature scoped against a job that fires "a few times a year" has a wildly different calls-per-user number than one scoped against a daily habit, and that number is the single biggest lever in the entire model.

The Scaling Curve: Why Cost Per User Grows, Not Just the User Count

Cost per inference doesn't scale linearly with users, because the average cost per call itself tends to rise as a feature matures — a second, compounding curve most launch models miss entirely. Growth in users is the curve everyone plans for; growth in tokens-per-call and calls-per-user is the one that quietly erodes margin underneath it.

Three forces reliably push the per-call number upward after launch:

• Context creep. Product teams add "smarter" context over time — retrieved documents, chat history, prior sessions — and each addition raises input tokens per call, often invisibly, since it happens through the RAG pipeline rather than a visible prompt change.
• Engagement compounding. Power users don't use a feature at the same rate as day-one testers; as the feature becomes habitual, calls-per-user climbs, and the most engaged users — the ones you're happiest to keep — are also the most expensive.
• Output creep. Richer formatting, structured JSON, multi-step reasoning chains, and self-correction passes all add output tokens, and some architectures make multiple model calls per user-visible action without anyone updating the cost model to reflect it.

How much context genuinely earns its keep, versus context that's padding the prompt out of habit, is itself a scoping question — and one worth running through a proper data readiness audit before deciding what a call is even allowed to retrieve. Every token you decide to feed the model is a token you're paying for on every single call, forever, at whatever scale the feature reaches.

Researchers tracking the field, including Epoch AI's ongoing analysis of AI compute trends, have documented that cost per token for a given quality level has fallen sharply over time — directionally on the order of a 5–10x drop across recent years, as smaller and more efficient models catch up to older frontier ones. That's a genuine tailwind, but not a substitute for modeling your own curve: usage growth and context creep can outrun the price decline, especially in the first year after launch, while a feature is still finding its habitual usage pattern.

Worked Example: How a Generous Free Tier Goes Underwater at Scale

A feature that costs about $240 a month to run for 1,000 beta users can cost roughly $1.65 million a month at 1,000,000 users — not because the per-user math changed conceptually, but because both tokens per call and calls per user grew alongside the user count, compounding on top of it rather than staying fixed.

Take a hypothetical AI meeting-summarizer feature, offered free and "unlimited" to drive adoption.

At beta launch (1,000 users): each call sends roughly 8,200 input tokens (a meeting transcript plus instructions) and returns about 350 output tokens, using a frontier-tier model. At an illustrative $3 per million input tokens and $15 per million output tokens, that's about $0.03 per call. Beta users run it on roughly 8 meetings a month, so cost per user is about $0.24/month — a rounding error, and genuinely magical at this scale.

At scale (1,000,000 users): the feature became sticky enough that the team added a "reference your last five meetings" context feature, pushing input tokens to about 16,000 per call and output to 450. Usage also intensified — habitual users now run it roughly 30 times a month. Cost per call rises to about $0.055, and cost per user climbs to $1.65/month.

If only 3% of that free base converts to a $12/month paid plan, that's 30,000 paying users generating $360,000/month in revenue — against $1.65 million/month in inference cost on the free tier alone. The feature that felt free to build at 1,000 users is now burning well over a million dollars a month at scale, before a single support ticket or infrastructure cost gets added on top.

An AI feature that's magical at 1,000 users can be bankrupt at a million. The formula didn't change between the two — the tokens-per-call and calls-per-user inputs to it did.

Four Levers That Fix the Model: Caching, Tiering, Batching, and Gating

None of the four standard levers require rebuilding the feature — they require deciding, deliberately, which calls actually need frontier-model quality and which don't, then routing accordingly. Applied together in the worked example above, they cut the monthly inference bill by roughly 94% without removing the feature's core value.

Prompt caching

Recurring meetings, repeated participant lists, and standing agendas mean a large share of input tokens are identical call to call. Both Anthropic and OpenAI publish prompt-caching pricing that discounts repeated context tokens steeply — directionally in the 50–90% range depending on provider and cache-hit pattern — which is close to free money for any feature with repetitive context.

Model tiering

Not every call needs the most capable model available. Routing straightforward, single-speaker, or short calls to a smaller model — something in the GPT-4o mini or Claude Haiku class rather than a frontier-tier model — and reserving the expensive model for genuinely complex cases can cut blended cost per call by an order of magnitude. Chip Huyen's writing on LLM system design makes the same case from an engineering angle: most production traffic doesn't need the best model available, only the cases that fail on a cheaper one do.

Batching

For anything that doesn't need a synchronous response — nightly digests, bulk re-summarization, backfill jobs — batch APIs (OpenAI's Batch API is one example) typically offer roughly a 50% discount in exchange for turnaround measured in hours rather than seconds. It's a lever that costs nothing but a scheduling decision.

Gating

Capping free-tier calls per month, and reserving expensive context features (like the "reference your last five meetings" example above) for paid seats, directly caps the worst-case cost per user. This is a pricing and product decision as much as an engineering one, and it interacts with where in the product a call happens — mapping the feature against a customer journey helps identify which touchpoints justify frontier-model spend and which are fine on a cheaper tier or a stricter cap.

Applied together — a blended small/frontier model mix, caching on repeated context, and a free-tier cap of 12 calls a month with extended context reserved for paid seats — the same worked example drops from $1.65 to roughly $0.10 in cost per free user per month, and total monthly inference cost at 1,000,000 users falls from $1.65 million to roughly $100,000.

The paid tier, now explicitly priced to cover its own higher-context inference cost (roughly $1.65/user against $12/month of revenue), stops subsidizing the free tier and starts carrying its own weight — the actual fix isn't "make AI cheaper," it's "stop paying frontier-model prices for calls that don't need them."

Treating Cost as a Feasibility Input, Not a Launch-Day Surprise

Cost per inference belongs in the same specification conversation as latency, effort, and data availability — evaluated before a launch date is committed to, not reconciled against the first month's bill after the feature ships. Feasibility, for an AI feature, is a four-way negotiation: what it costs, how fast it responds, what it takes to build, and what data it needs, all traded off together rather than one at a time.

That's also why model tiering isn't a free lunch: swapping in a smaller model to cut cost changes more than the invoice — it changes the response profile too, and that trade-off deserves the same scrutiny covered in latency budgets for AI features. A cheaper model that's also slower, or a cached response that's gone stale, can cost you in churn what it saves in tokens.

It's also why cost estimation shouldn't live only with engineering after a PM has already promised a feature "unlimited" and "free." It's one of the concrete inputs raised in the six questions engineering needs answered before scoping an AI feature — asked at spec time specifically because it's cheap to change a token budget or a tiering rule on paper, and expensive to change it after users have formed a habit around an unmetered feature.

Key Takeaways

• Marginal cost is not zero for AI features — every call consumes metered tokens or compute, which changes the economics of scale from "grows for free" to "grows the bill."
• The model has three moving parts: tokens per call, calls per active user, and price per token — multiply them together and by your user base to get a real monthly number.
• Cost per call rises after launch, not just user count — context creep, engagement compounding, and output creep all push the per-call number up over time.
• A generous free tier is a cost decision, not just a growth lever — model it against a specific calls-per-user ceiling before calling it "unlimited."
• Four levers — caching, model tiering, batching, and gating — can cut inference cost by an order of magnitude without removing the feature's core value, if applied deliberately rather than after the bill arrives.
• Cost, latency, effort, and data readiness are one feasibility conversation, not four separate ones — treat per-inference economics as a spec-time input, alongside the framework in Prodinja's Feature-to-Feasibility tool.