AI feature estimates break because the deliverable itself is unknown until you test it against real data and real users — you're pricing an experiment, not a build. The fix is to stop naming one number and instead size three phases separately: a feasibility spike, a first working version, and hardening to a real bar, with a kill-gate in between.
Quick Answer: Estimate AI features in three parts — a timeboxed feasibility spike (proves it's possible), a first working version (proves it solves the job), and hardening-to-bar (the long tail of accuracy, latency, and edge cases). Put a kill-gate after the spike so uncertainty gets a decision, not a due date.
Why Story-Point Math Fails the Moment AI Enters the Backlog
Story points assume you already know the shape of the work — you're sizing a known solution against past team velocity. AI features break that assumption because the solution's behavior, not just its size, is unknown until you run it against real inputs; you cannot size what you haven't yet observed.
A CRUD feature is deterministic. Given a spec, a login form or a reporting screen behaves the same way every time it's built correctly, so effort scales predictably with the number of screens, fields, and integrations. An AI feature's "correctness" is a statistical property — precision, recall, hallucination rate, p95 latency — that only exists once the thing is actually running against your data. You cannot inspect a spec and know it; you can only measure it.
This isn't a metaphor. Researcher Dave Snowden's Cynefin framework distinguishes complicated problems (the right answer is knowable in advance through expert analysis) from complex ones (the right answer only emerges by probing, sensing, and responding). Most CRUD work is complicated — plannable. Most AI feature behavior is complex — you learn what's true only after you try. Planning harder doesn't fix a complex problem; running a small, cheap experiment does.
| Dimension | CRUD feature | AI feature |
|---|---|---|
| What "done" means | Matches an agreed spec | Clears a statistical bar (precision, recall, latency) discovered mid-project |
| Main cost driver | Number of screens, fields, integrations | Number of edge cases and iterations needed to hit the bar |
| Estimation method | Compare to similar past tickets | Compare to similar past experiments — a different reference class entirely |
| Confidence over time | Roughly flat once scoped | Low at kickoff, jumps after the spike, plateaus during hardening |
| Primary risk | Scope creep | Discovering the task has a lower achievable ceiling than assumed |
The Standish Group's long-running CHAOS research on software delivery has repeatedly found that the highest-uncertainty, highest-complexity projects are also the ones most likely to blow past their original budget and timeline by a wide margin — often multiples of the original estimate, not a modest overrun. AI features sit squarely in that high-uncertainty bucket.
That's exactly why treating them like a normal backlog item is where the estimate starts breaking. For a broader framework on scoping this kind of work before it hits a sprint, see our complete guide to AI feasibility.
Estimate the Experiment, Not the Deliverable
The core shift is separating the learning phase (does this even work, and how well?) from the build phase (turn a validated approach into a shipped feature). Estimate them with different units and different confidence: learning gets a timebox and a decision; building gets a normal estimate — but only once learning has actually reduced the uncertainty.
Inside "does this work," split further into unknowns you can resolve and unknowns you can't:
- Reducible unknowns — things a spike can answer: does the model handle our messy, real-world data; what's the actual
p95 latencyunder production load; what's the false-positive rate against real user inputs, not a curated demo set. - Irreducible unknowns — the task's intrinsic noise ceiling. If two trained human reviewers disagree with each other 15% of the time on the same judgment call, no model can score meaningfully above that agreement ceiling — a concept close to what statisticians call the
Bayes error rate. No amount of extra iteration closes this gap; it's a property of the task, not the tool.
Confusing the two is where estimates quietly go wrong. Teams keep iterating on an irreducible gap as if one more prompt tweak will close it, burning sprints that a spike should have flagged as a ceiling, not a bug.
A large share of the reducible category, in practice, turns out to be data quality, not model choice. Messy, inconsistent, or sparse source data is usually the first thing a spike should stress-test — exactly the gap our AI feature data readiness audit is built to catch before it derails a project.
Author Douglas Hubbard's work on calibrated estimation (How to Measure Anything) makes the same case for any high-uncertainty forecast: replace a false-precision point number with a calibrated range — something closer to a 90% confidence interval than a single date. Applied here, that means the honest estimate for an AI feature at kickoff isn't "6 weeks," it's "2 weeks to know if this is viable, then somewhere between 4 and 10 weeks depending on what we learn."
A quick self-check for whether a feature's estimate is already broken before a single line of code is written:
- The acceptance criteria describe a UI flow, not a measurable behavior threshold.
- Nobody has said out loud what accuracy, latency, or error rate would count as "good enough."
- The estimate was handed over before anyone ran the model against real production-shaped data.
- Engineering was asked for a single date rather than a range with a named checkpoint.
If two or more of those are true, you're estimating the wrong thing.
The Three-Part Estimate: Spike, First Working Version, Hardening to Bar
Split the work into a feasibility spike that proves the idea is viable, a first working version that proves it solves the actual job end-to-end, and a hardening-to-bar phase that closes the gap between demo-quality and production-quality. Estimate each phase separately — their effort, risk profile, and even the right people in the room all differ.
1. Feasibility spike
Borrowing the term from Kent Beck's Extreme Programming practice, a spike is a strictly timeboxed piece of work whose only job is to answer a specific question — never to produce shippable code. For an AI feature, the question is usually some version of: "can this approach clear an acceptable bar on our real data, inside our real constraints?"
One senior engineer or ML-capable practitioner, working with the PM to define the pass/fail line, is usually enough. Before this starts, engineering needs a small set of concrete answers from product — the kind covered in the six questions engineering needs before an AI spec — so the spike is testing the right hypothesis instead of a vague one.
2. First working version
Once the spike says "viable," the first working version (FWV) is a thin, deliberately unpolished, end-to-end slice that a real user could actually try on one real scenario. It is not the product — it's proof that the validated approach, wired into the actual product surface, solves the job it's meant to solve. This is where you find integration friction the spike never surfaced: how the feature behaves inside the actual UI, under actual latency conditions, with actual upstream data.
3. Hardening to bar
This is almost always the largest and most underestimated bucket. Hardening is the work of closing the gap between "worked in the demo" and "meets the bar we're willing to ship broadly" — accuracy across the real distribution of inputs, latency budgets under real load, safety and abuse cases, monitoring, and graceful failure. Because a demo only has to work once, in front of a friendly audience, teams routinely mistake FWV for done and then get blindsided by how much hardening remains.
| Phase | Central question it answers | Typical share of total effort* | Who needs to be in the room | What "done" looks like |
|---|---|---|---|---|
| Feasibility spike | Can this work at all, on our data, inside our constraints? | 10-20% | One senior engineer/ML practitioner + PM | A go/no-go recommendation and a refined range — not a shippable artifact |
| First working version | Does this solve the actual job end-to-end, for one real scenario? | 25-35% | A small build squad | A thin, unpolished, end-to-end path a real user could try |
| Hardening to bar | Does it meet the accuracy, latency, safety, and edge-case bar to ship broadly? | 45-60% | Full team, plus QA/security/legal as needed | Meets the pre-agreed bar across the defined edge-case set |
*Illustrative ratios, not a formula — directionally consistent with the pattern Google's internal Rules of Machine Learning guidance (Martin Zinkevich) describes: most of the durable effort in a shipped ML system goes into data, infrastructure, and monitoring, not the initial model itself.
What counts as "shippable" during hardening should also account for where in the experience the feature sits. A bar for a nice-to-have surface is legitimately looser than one for a moment mapped on your customer journey as high-stakes or high-friction, and part of hardening is confirming the feature meets latency budgets appropriate to that moment — a topic our piece on latency budgets for AI features covers in more depth.
The Kill-Gate: Give the Spike's Uncertainty a Decision, Not a Deadline
A kill-gate is a decision point immediately after the spike where the team explicitly chooses to proceed, narrow scope, redesign, or stop — using criteria agreed before the spike started, not invented after seeing the results. Without a pre-committed gate, "this is harder than we thought" quietly turns into "let's just add another sprint," and the sunk-cost spiral begins.
The gate only works if the pass/fail numbers are written down in advance — before anyone has an emotional stake in a particular answer. Four honest outcomes should all be on the table when the spike concludes:
- Proceed as planned — the spike confirms the approach clears the bar; move into the first working version with the original range intact.
- Proceed with narrower scope — the spike shows the model handles a meaningful subset of cases well but not all of them; ship that subset first and treat the rest as a later iteration.
- Redesign the approach — the spike shows the specific technique tried (say, a single prompt) can't clear the bar, but a different architecture (retrieval-augmented, a fine-tune, a rules-plus-model hybrid) might.
- Kill or shelve it — the spike reveals an irreducible ceiling below what the job actually requires. The professionally uncomfortable but correct move is to not build it.
That fourth option is the one teams are worst at choosing, which is precisely why it needs to be pre-authorized rather than argued for in the moment. Analyst firm Gartner has publicly and repeatedly flagged that a substantial share of AI proof-of-concepts never reach production — the exact failure mode a kill-gate is meant to make cheap and early instead of expensive and late.
The "bar" the gate checks against should trace back to a real user job, not an arbitrary accuracy target picked in a planning meeting. Grounding the feature in jobs-to-be-done analysis earns its keep here: the acceptable error rate for "help me draft a first pass" is very different from the one for "tell me whether to escalate this."
The kill-gate's real function isn't stopping bad projects — it's making the stop-or-continue decision boring and pre-agreed, instead of a political fight that happens under deadline pressure.
Communicating a Range Without Sounding Like You Don't Know Your Job
Give stakeholders a range anchored to a checkpoint, not a single date: "two weeks to know if this is viable, then a range of four to ten weeks depending on what we learn." A range with a named decision point reads as rigor. A single confident date that later slips reads as incompetence — even though the AI feature was always the riskier bet.
In practice, that means restructuring the conversation stakeholders expect to have:
- Name the spike's exact question and timebox up front — "by [date], we'll know whether precision clears 85% on real support tickets," not "we're exploring AI for support."
- Give the range with its assumptions attached, so a stakeholder sees why it's a range: "4-10 weeks, wide because we don't yet know how much hardening the edge cases will need."
- Report the kill-gate outcome as a real update, even when it's "we're narrowing scope" or "we're stopping." A gate that only ever says "proceed" isn't a gate.
- Re-tighten the range after each phase, the same way a calibrated forecaster narrows a confidence interval as new evidence arrives — the range should visibly shrink as the project proves things out, not stay static until it snaps.
This is a more honest use of the same estimation muscle PMs already apply to prioritization frameworks like RICE or Kano — the point was never a false-precision score, it was a structured, defensible way to reason under uncertainty and be able to explain the reasoning later.
Where Prodinja Fits: Turning "How Long Will This Take" Into a Structured Range
This entire shift — reducible versus irreducible unknowns, a spike before a date, a range instead of a point estimate — is exactly what Prodinja's Feature-to-Feasibility tool is built around. Rather than asking you to name a single number up front, it's designed to walk you through separating the unknowns a spike can actually resolve from the ones that are inherent to the task, so what comes out the other end is a range paired with a concrete de-risking spike, not a confident-sounding date nobody can defend later.
That framing matters more than the specific numbers it produces. An estimate that's honest about what it doesn't yet know is more useful to an engineering lead and a stakeholder alike than one that's precise-looking and wrong — and it's designed to hand the spike's output straight into the same working document engineering and design keep updating as the project moves from "can this work" to "does this meet the bar."
Key Takeaways
- AI feature effort is a probability distribution, not a story point — the deliverable's behavior is unknown until it's run against real data, so sizing it like a known-shape CRUD ticket is the root cause of most AI estimate overruns.
- Split learning from building. A feasibility spike answers "can this work"; a first working version answers "does this solve the job"; hardening-to-bar answers "is it good enough to ship broadly" — estimate all three separately.
- Separate reducible unknowns (a spike can resolve them) from irreducible ones — an intrinsic task ceiling, close to a
Bayes error rate, that more iteration can't close. - Put a kill-gate right after the spike, with pass/fail criteria written down before the spike starts, so "proceed," "narrow scope," "redesign," and "kill it" are all genuinely live options.
- Hardening to bar is usually the largest, least-estimated bucket — budget for accuracy, latency, safety, and edge cases, not just getting a demo to work once.
- Communicate a range anchored to a named checkpoint, not a single date — and re-tighten that range as each phase actually reduces uncertainty.
- Ground the "good enough" bar in the real job the feature serves, not an arbitrary accuracy number picked in a planning meeting.
Frequently Asked Questions
How do you estimate AI feature effort when you don't know if it will work?
You don't estimate the whole feature as one number — you estimate a short, timeboxed feasibility spike first, then re-estimate the rest once the spike has actually reduced the uncertainty. The spike's job is to convert "we don't know" into evidence a real range can be built on.
Why do AI projects consistently run over budget and schedule?
Most overruns come from treating a probabilistic, complex problem like a plannable, complicated one — sizing the AI feature the way you'd size a CRUD ticket, then discovering mid-project that the real cost driver is hardening to an accuracy and latency bar nobody named up front. Analyst research from firms like Gartner has repeatedly pointed to a large share of AI pilots stalling before production for exactly this reason.
What is a feasibility spike in AI development, and how long should it take?
A feasibility spike is a strictly timeboxed piece of exploratory work — commonly one to two weeks — whose only goal is answering a specific pass/fail question about whether an approach can clear an acceptable bar on real data. It should never be scoped as "however long it takes to make it good," since that erases the timebox that makes it useful.
Should AI features ever be estimated in normal story points?
Once hardening-to-bar begins and the approach is validated, the remaining work often does behave more like conventional engineering and can be sized with normal estimation methods. The spike and first-working-version phases, by contrast, are experiments with an unknown outcome and are better sized as a timebox plus a decision than as story points.
What happens if the feasibility spike fails?
A failed spike is a successful outcome of the process, not a failed project — it's exactly the cheap, early signal a kill-gate exists to produce. The honest options are narrowing scope to what did work, trying a different technical approach, or shelving the idea, all of which are far less costly than discovering the same ceiling after months of full-team build effort.