Buy, Build, Fine-Tune, or Prompt? The Four-Way Feasibility Call for AI Features

The right call is almost never "build a custom model." Start with a vendor API and prompt engineering; move up only when a specific, provable constraint—differentiation, a data moat, cost at scale, latency, or privacy—makes that rung impossible. The same "summarize documents" feature can cost a week or a quarter depending entirely on which rung you pick first, so the decision deserves more rigor than "what did the last team do."

Quick answer: Default to vendor API + prompt. Climb to fine-tune only when prompting can't hit your accuracy, latency, or unit-cost target. Climb to build only when fine-tuning can't solve a real differentiation, data-moat, or infrastructure constraint. Each rung up should be a forced move, not a preference.

The Four-Rung Ladder: From Vendor API to Custom Model

Four implementation strategies sit on one ladder, ordered by effort and control: buy + prompt, buy + fine-tune, open-weight + fine-tune, and build. Each rung up trades a week of engineering for a quarter, and trades low control for a lot more of it—plus a lot more that can go wrong.

Think of it less as four separate options and more as a single spectrum you walk along only as far as you're forced to:

The jump from Rung 1 to Rung 2 is usually incremental. The jump from Rung 2 to Rung 3 is where most teams underestimate cost, because it adds a whole MLOps discipline—GPU provisioning, model versioning, drift monitoring—that a vendor API quietly handled for free. Rung 4 is a different company decision, not a sprint decision; it belongs in your complete guide to AI feasibility rather than a single feature spec.

One Feature, Four Builds: Summarizing Documents at Every Rung

The same feature request—"let users get a summary of an uploaded document"—produces four wildly different builds depending on the rung, and none of them are wrong in isolation. What's wrong is picking a rung the feature's actual constraints don't justify.

Rung 1 — Buy + Prompt. You send the document text (or chunks of it, if it's long) to a vendor API with a well-crafted system prompt: summarize for X audience, in Y tone, hitting Z structure. You iterate on the prompt, add a few output examples, and maybe add basic retrieval so the model only sees relevant sections. Shippable in days; quality is "good enough" for most generic documents.

Rung 2 — Buy + Fine-Tune. Generic summaries are fine until your documents aren't generic—contracts, clinical notes, engineering RFCs, each with a house style and a required structure the prompt keeps almost-but-not-quite hitting. You fine-tune a vendor model on a few hundred to a few thousand examples of your ideal summary, so the format becomes reliable instead of prompted-for. This is the rung where a genuine data readiness audit starts to matter, because fine-tuning is only as good as the labeled examples you feed it.

Rung 3 — Open-Weight + Fine-Tune. Now assume summarization sits on the critical path of the product's most valuable moment in the customer journey—say, the moment a lawyer decides whether to trust the tool at all—and your legal team won't let a single page of client documents leave your infrastructure. You fine-tune an open-weight model you host, trading a vendor's convenience for data residency and full control over the serving stack.

Rung 4 — Build. Finally, imagine summarization isn't a feature at all—it's the entire product, and your edge is a proprietary architecture or a training corpus nobody else has (legal case outcomes, years of internal engineering docs). Here you're not fine-tuning someone else's model; you're building the thing competitors can't buy off a shelf. Very few teams should ever reach this rung for a single feature.

Same Feature, Four Very Different Bills

The Five Signals That Push You Up the Ladder

Five signals — and only these five, in practice — justify climbing a rung: differentiation, data moat, cost at scale, latency, and privacy. Absent a clear, currently-true answer on at least one of them, the honest default is to stay put.

Each signal maps to a different failure mode if you ignore it, which is why they're worth checking one at a time rather than as a vague gut feeling.

1. Differentiation. Is this feature the actual job the customer is hiring your product to do, or a supporting step around it? A feature you'd defend as core to your jobs-to-be-done deserves more investment than a utility feature every competitor also ships.
2. Data moat. Do you have proprietary, labeled data that compounds — more usage produces better data produces a better model — or would you be fine-tuning on the same public data everyone else has access to? No moat, no reason to fine-tune yet.
3. Cost at scale. Does the vendor's per-token price, multiplied by your realistic volume, threaten unit economics? Cost pressure is real, but model it against actual usage projections, not a worst-case guess.
4. Latency. Can the feature tolerate a network round-trip to a hosted API, or does it sit inside a real-time interaction where every added millisecond costs conversion? Latency budgets should be set before this question is answered, not after.
5. Privacy and regulatory constraint. Is there a contractual, legal, or regulatory reason data cannot leave your infrastructure at all — not "would prefer not to," but "cannot"? This is the one signal that can force a jump straight to Rung 3, skipping fine-tuning-on-vendor entirely.

Signals That Feel Real But Usually Aren't

Four objections show up in almost every roadmap debate and sound like the five signals above without actually being one of them. Naming them shortens the argument considerably.

• "Competitors are fine-tuning, so we should too." Competitor behavior isn't a constraint on your feature — it's a rumor. Ask what signal they had, not what they did.
• "It'll probably need this eventually." Eventually isn't now, and vendor models improve monthly; a constraint you can't measure today shouldn't be solved today.
• "Our engineers want to work with open models." A legitimate learning goal, but it's a staffing decision, not a feasibility one — don't let it borrow the feature's budget.
• "The demo would be more impressive." Impressiveness is not on the list. If it doesn't map to differentiation, moat, cost, latency, or privacy, it's scope creep wearing a feasibility costume.

Why You Should Start at the Top and Descend Only Under Real Constraint

The instinct to build or fine-tune first is usually ego or architecture-astronaut thinking, not a feasibility conclusion — most feature ideas resolve perfectly well at Rung 1, and jumping ahead just adds months you can't get back. Treat every rung above the first as something you have to justify with evidence, not assume as a starting point.

This isn't a contrarian take; it's close to consensus among the people who build these systems for a living. Andreessen Horowitz's "Emerging Architectures for LLM Applications" report found that most production LLM applications rely on retrieval-augmented prompting rather than fine-tuned weights, since retrieval is faster to iterate and debug.

OpenAI's own fine-tuning documentation makes the same case: exhaust prompt engineering and retrieval first, and reserve fine-tuning for cases where you need a consistent output format or style the prompt can't reliably enforce.

That "format, not facts" framing has a well-known source: Andrej Karpathy has described fine-tuning as good for teaching a model how to respond — tone, structure, house style — while retrieval and context are the right tool for teaching it what to know. Confusing the two is the single most common reason a fine-tuning project underdelivers: teams fine-tune hoping to inject new facts, when what they needed was better retrieval.

Descending the ladder should feel like admitting defeat at the rung below, not like an ambitious roadmap decision.

The cost of skipping ahead compounds too. Gartner has predicted that a substantial share of generative-AI projects — its analysts put the figure at roughly a third — would be abandoned after proof-of-concept, citing escalating cost and unclear business value as leading causes.

Chip Huyen, in her writing on applied AI engineering, makes a related point: model choice is rarely the real bottleneck. Data quality, evaluation rigor, and the surrounding product decisions matter more, and all three get harder the higher you climb before you've earned it.

The Three Questions That Actually Decide the Rung

• Have we genuinely exhausted the rung below? Not "did we try one prompt" — did we iterate the prompt and retrieval design the way we'd iterate any other feature, with real evals against real examples?
• Is the constraint measured, not assumed? A latency budget, a cost-per-request ceiling, or a signed data-processing agreement is evidence. "It feels slow" or "we might need this later" is not.
• Would we make the same call in six months? Vendor models improve quickly; a fine-tuning project justified today by a capability gap can become unnecessary by the time it ships.

Running the Call: A Decision Framework and Where Prodinja Fits

Run the call as a short, explicit exercise before a single line of model-facing code gets written: name the feature, name the signal you believe forces a rung above the floor, and name the evidence for it. If no signal survives that scrutiny, ship Rung 1 and revisit later — reversibility is cheap this early and expensive once fine-tuned weights or custom infrastructure are load-bearing.

A workable sequence, in order:

1. Ship the Rung 1 version first, even if you're fairly sure you'll need more — it's your cheapest possible eval harness, and it tells you exactly where prompting actually falls short instead of where you assumed it would.
2. Log the failure mode, not just the failure rate. "Wrong facts" points toward retrieval; "wrong format 30% of the time" points toward fine-tuning; "too slow for the interaction" points toward latency work, possibly a smaller hosted model before a custom one.
3. Match the failure mode to one of the five signals, in writing, and name the evidence. If it doesn't match one, you have a backlog item, not a feasibility case.
4. Re-run the check before each major release, since the constraint that justified a rung six months ago may have quietly resolved itself as vendor models got better or cheaper.

This is also exactly the kind of decision that should show up in the spec engineering sees, not get discovered mid-sprint — the six questions engineering needs before an AI feature spec is a useful checklist to pair with the rung you land on, since "which rung" answers several of those questions at once (latency, cost, data handling).

Prodinja's Feature-to-Feasibility tool is built around this exact tension. In the prototype, it frames the model-approach choice as a spec decision, not an afterthought — walking through control, cost, and effort tradeoffs across vendor API, prompt, fine-tune, and build so the reasoning is visible and reviewable before anyone commits engineering time to a rung the feature never actually needed.

Key Takeaways

• Default to Rung 1 (vendor API + prompt). It's the cheapest, fastest, and most reversible option, and it resolves most feature requests without further debate.
• Climb only on evidence, not preference. Differentiation, data moat, cost at scale, latency, and privacy are the five signals that legitimately force a move up the ladder — a gut feeling isn't a sixth.
• Fine-tuning fixes form, not facts. Use it to lock in tone and structure; use retrieval and context engineering to inject knowledge, per Karpathy's framing and OpenAI's own guidance.
• The MLOps tax hits hardest between Rung 2 and Rung 3. Hosting your own weights adds a discipline (versioning, drift monitoring, GPU provisioning) a vendor API handled invisibly.
• Privacy is the one signal that can skip a rung entirely, forcing a direct jump to self-hosted fine-tuning when data legally cannot leave your infrastructure.
• Revisit the call periodically. A capability gap that justified fine-tuning today can close as vendor models improve, per Gartner's and a16z's shared observation that most production complexity migrates toward simpler architectures over time, not away from them.
• Write the rung into the spec, not just the code — it answers cost, latency, and data-handling questions engineering will otherwise have to reverse-engineer later.