A Feasibility Scoring Rubric for Triaging a Backlog of AI Feature Ideas

A feasibility scoring rubric assigns each AI feature idea a 1-5 rating across five dimensions — data readiness, error tolerance, effort, cost-at-scale, and maintenance — then sums them into one number. That number lets you rank a backlog of AI requests the same way you'd rank anything else, instead of eyeballing each idea in isolation.

Quick answer: Score every AI feature idea 1-5 on data readiness, error tolerance, effort, cost-at-scale, and maintenance. Sum the five scores into one feasibility number, then feed that number — not a gut feel — into RICE or Kano to rank the backlog.

Why "Gut Call" Feasibility Breaks Down at 20 Requests

Gut-call feasibility works when one PM eyeballs one feature. It collapses once ten or twenty AI requests compete for the same sprint, because nothing forces a consistent comparison — whoever pitches loudest, most recently, or with the most executive weight behind them wins, regardless of whether their idea is actually buildable at acceptable risk and cost.

Three biases show up in almost every backlog review, and none of them correlate with whether an idea will actually work in production:

• Recency bias — the request from this morning's leadership meeting feels more urgent than one that's sat in the backlog for a month, independent of merit.
• Seniority bias — a VP's "can we just add AI to this" gets scoped ahead of a support lead's smaller, better-understood request.
• Novelty bias — "agentic" and "autonomous" framing reads as more valuable than "assistive," even when the assistive version is safer and cheaper to ship.

None of this is a people problem — it's a tooling gap. Without a shared scale, feasibility conversations default to whoever argues best in the room. Gartner has predicted that a large share of agentic AI projects — its 2025 forecast put the figure above 40% — will be abandoned by 2027, citing unclear business value and rising cost as the leading causes, not a failure of the underlying models. A rubric exists to catch that gap before a team spends a quarter building the wrong idea.

A feasibility score doesn't replace judgment about desirability or reach — it isolates the one variable stakeholders are worst at estimating for themselves: can this actually be built, run, and kept correct at the volume the business needs? For the fuller picture of how feasibility fits into the rest of an AI roadmap, see our complete guide to AI feasibility.

The Five-Dimension Feasibility Rubric

The rubric scores five dimensions on a 1-5 scale, where 5 is the most feasible outcome: data readiness, error tolerance, effort, cost-at-scale, and maintenance. Summing the five scores produces one comparable number for every idea in the backlog, whether it's a small chatbot tweak or a fully autonomous agent.

Each dimension answers a question engineering already asks privately during scoping — the rubric just makes the answer visible and comparable before the planning meeting, instead of during it or after the sprint has started.

Data Readiness: Does the Data Already Exist?

Data readiness measures how much of the data an idea needs is already collected, labeled, and accessible versus theoretical. A feature that reuses data flowing through production today scores near 5; one that requires new instrumentation, a data-sharing agreement, or manual labeling from scratch scores near 1.

This is usually the fastest dimension to score wrong, because a stakeholder assumes "we have that data somewhere" without checking with the team that owns it. That's the exact gap a structured feasibility interview with an ML engineer is built to close before the score gets written down.

Error Tolerance: What Happens When It's Wrong?

Error tolerance measures the blast radius of a wrong output, not how often the model is likely to be wrong. A drafting tool a human reviews before it ships scores high; an autonomous action with financial, legal, or safety consequences scores low, no matter how accurate the model tests at in isolation.

Where the mistake lands in the experience matters as much as the mistake itself. A wrong answer at a moment of high emotional stakes, the kind mapped in a customer journey emotion curve, costs more trust than the identical error at a low-stakes moment.

Effort: What Does a Production Version Actually Take?

Effort scores the engineering complexity of a production-grade version, not a demo. A working prototype in a notebook is not evidence of a 4 or 5 here — production means error handling, monitoring, and integration with existing systems, which is usually where estimates quietly double. Running the idea through the six questions engineering needs before scoping an AI spec before scoring keeps the number honest instead of optimistic.

Cost-at-Scale: What Does It Cost at Full Volume?

Cost-at-scale scores the per-use inference and infrastructure cost projected at expected rollout volume, not the cost of ten test calls in a demo. A feature that calls a large model with a long context window on every request can look free in a pilot and become the single largest line item in the infrastructure budget at scale. Cost and latency are usually the same conversation — working out a latency budget for an AI feature forces the same volume-and-model-size questions that expose the real cost curve.

Maintenance: What Does It Take to Keep It Correct?

Maintenance scores the ongoing work required to keep a feature accurate as data, user behavior, and the underlying model provider all change. A rules-based classifier with a stable input distribution scores high; anything needing scheduled retraining, drift monitoring, or a human-in-the-loop review queue to stay safe scores low. This is the dimension most backlogs skip entirely, because it doesn't show up until two quarters after launch — by which point it's a support-team problem, not a product one.

The table below condenses each dimension's low, middle, and high anchor so a team can score consistently without re-deriving the scale every planning cycle.

Score every idea against this same scale. An idea that scores 5 on data readiness but 1 on error tolerance isn't "average feasibility" — it's a specific, nameable risk that a single blended number would otherwise hide.

Scoring the Backlog: A Worked Example

Scoring a real backlog means running four or five competing AI ideas through the same five questions and summing the results into a total from 5 to 25. Anything above roughly 18 is buildable now, 12-17 needs rework or a narrower scope, and below 12 is a kill-or-radically-rescope signal.

Below is a representative backlog with four ideas scored side by side — the kind of mix a support or growth team might actually be triaging in one planning cycle.

The pattern that matters most: the two ideas with the highest apparent business upside — instant refund approval and real-time pricing — score lowest. Both are seductive-but-infeasible: easy to pitch in a sentence, expensive and risky to actually run unattended. The rubric doesn't say "never build these"; it says build the human-reviewed, reversible version first, and revisit full autonomy once error tolerance or data readiness genuinely improves.

Notice, too, that the top-scoring idea isn't the most ambitious one — it's the one where a human stays in the loop. Keeping a person in the approval path is often the single change that moves error tolerance, and therefore the total score, the most for the least engineering effort.

Running the Scoring Session Without It Turning Into a Debate

Score independently before discussing as a group, or the loudest voice in the room anchors everyone else's numbers before they've thought it through. Have each participant — PM, engineer, and data owner — write down all five scores privately first, then compare.

1. Timebox each idea to 10-15 minutes, including the discussion of any score gap between participants.
2. Discuss disagreements, not agreements — if everyone lands on a 4 for effort, move on; if scores range from 1 to 4, that gap is the most useful information in the room.
3. Write down the reasoning next to the number, not just the score itself, so next quarter's re-score isn't starting from a blank page.

A score with no rationale attached decays into a gut call with extra steps within two review cycles — the rationale is what makes a rescore later a genuine update instead of a re-argument from scratch.

From Feasibility Score to Rank: Feeding RICE or Kano

A feasibility score isn't a priority score by itself — it's one input that makes RICE's effort estimate and confidence multiplier more honest, or tells Kano which features can even be table stakes versus which only work as a delighter if the model is reliable enough to trust unsupervised.

RICE (Reach, Impact, Confidence, Effort) already asks for an effort estimate and a confidence discount — the feasibility rubric is where those two numbers should actually come from, instead of one engineer's hallway guess. A low error tolerance score is a legitimate reason to discount RICE confidence; a low effort score (meaning high complexity) is a legitimate reason to raise the RICE effort denominator, which lowers the resulting priority automatically.

Kano categorization asks whether a feature is a basic expectation, a performance feature, or a delighter — and an AI feature's category can flip entirely based on its feasibility profile. An AI summary that's occasionally wrong is a delightful nice-to-have; the same feature marketed as the system of record becomes a basic-expectation failure the moment it's wrong in front of a customer. Scoring feasibility first keeps the Kano conversation grounded in the feature as it will actually ship, not as it was pitched.

Before feeding a score into either framework, check that the "value" side of the equation is actually a customer's job to be done, not a manager's hunch. A quick pass through the customer's jobs to be done keeps the impact side honest the same way the rubric keeps the effort side honest.

Don't Score Hype as Value — Common Rubric Mistakes

The most common failure isn't scoring one dimension wrong — it's letting hype leak into the value side of the equation so an unfeasible idea gets waved through anyway. Score feasibility as a hard, separate axis from value, or the rubric becomes just a more elaborate way to greenlight whatever's fashionable this quarter.

1. Treating "it's AI" as value. A feature isn't more valuable because a model is involved — score the underlying job it does for the customer, then score feasibility separately.
2. Scoring the demo, not the production system. A prompt that works on five hand-picked examples tells you almost nothing about effort, cost, or maintenance at real volume.
3. Skipping maintenance because launch feels like the finish line. Retraining cadence and drift monitoring are real, recurring costs that belong in the score, not a surprise two quarters later.
4. Letting seniority override a low score. A rubric only holds comparability if a VP's pet idea gets the same five questions as everyone else's.
5. Never re-scoring after the prototype teaches you something. A first integration attempt is often the moment a "3" on data readiness quietly becomes a "1."

McKinsey's research on generative AI has repeatedly found that realized value concentrates in a handful of use cases — customer operations, marketing and sales, software engineering, and R&D — while the long tail of smaller ideas contributes disproportionately little, which is exactly the pattern a hype-inflated value score obscures.

Google's widely circulated Rules of Machine Learning, written by engineer Martin Zinkevich, opens with a related warning: rule one is "don't be afraid to launch a product without machine learning." A simpler, non-AI version of an idea is often the right score-5 answer, not a concession.

NewVantage Partners' long-running executive data survey has, year after year, found that most large organizations still name data-related and cultural obstacles — not model quality — as the primary barrier to realizing AI value, which is precisely what the data readiness dimension is built to catch early.

Keep the rubric a living document, not a one-time exercise. Score with a PM, an engineer, and whoever owns the data source in the room together, and re-score anything whose prototype surfaced a real surprise.

Key Takeaways

• A shared 1-5 scale beats a gut call once a backlog exceeds a handful of AI ideas — it removes recency, seniority, and novelty bias from the comparison.
• Five dimensions catch what one "is this feasible?" question misses — data readiness, error tolerance, effort, cost-at-scale, and maintenance each fail independently.
• Seductive ideas often score lowest on error tolerance, not effort — an autonomous, irreversible action can look cheap to build and still be a bad idea to ship unattended.
• Effort and cost should be scored for the production system, not the demo — pilot-scale numbers routinely understate both by a wide margin.
• Feasibility feeds RICE and Kano — it doesn't replace them — it makes their effort and confidence inputs honest instead of guessed.
• Rescoring after a prototype is the rubric doing its job, not overhead — treat a changed score as new information, not a failure of the first estimate.
• Hype is not a value input — score whether a feature does a real customer job separately from whether it happens to involve AI.