Most CPO-level calls have to be made with roughly 60-70% of the information you'd want, on decisions that are hard or impossible to reverse. The skill that separates strong executives isn't finding more data — it's classifying the decision correctly, forecasting its outcome with an explicit confidence level, and logging both so you can grade your judgment later.

Quick answer: Sort every high-stakes call into a Type-1 (one-way door) or Type-2 (two-way door) bucket using Jeff Bezos's framing, write down your predicted outcome and confidence before you act, then review the actual result against that prediction. Judgment, like any skill, only improves if you keep score.

From "Find More Data" to "Reason Well Under Uncertainty"

The instinct at every level below CPO is to ask for one more analysis, one more customer call, one more quarter of data. At the top, that instinct becomes a liability: the decisions that reach an executive's desk are, almost by definition, the ones nobody below could resolve with the data on hand. Waiting for certainty at that altitude isn't diligence — it's an expensive way to avoid owning a call.

Amazon founder Jeff Bezos made this explicit in his 1997 and 2015 shareholder letters: as a company scales, the senior team must develop a high-velocity decision-making muscle, because most of the decisions that matter most are made with incomplete information regardless of how long you wait. The data doesn't run out because your team failed to gather it. It runs out because the question is inherently under-determined — no amount of research fully resolves whether a market will accept a new pricing model or a competitor will retaliate.

This reframe matters because it changes what "good process" looks like. A PM optimizing a checkout flow can A/B test their way to certainty. A CPO deciding whether to sunset a legacy product line, enter a new geography, or restructure the pricing model cannot — the sample size is one, the feedback loop is measured in quarters, and a wrong call is expensive to unwind. That's a different discipline: calibrated judgment, not exhaustive research.

Three habits build that discipline, and the rest of this piece works through each in order:

  1. Classify the decision by reversibility before debating its merits.
  2. Write down a prediction and a confidence level before you commit — not after.
  3. Log the decision and its assumptions, then review the actual outcome against what you predicted.

None of this eliminates uncertainty. It converts uncertainty you can't remove into a skill you can measurably improve, the same way a weather forecaster gets better at "70% chance of rain" by checking, year over year, whether it actually rained 70% of the time they said so.

The Door Test: Sorting Type-1 from Type-2 Decisions

Bezos's Type-1/Type-2 framework asks one question before any other analysis: can you walk back through this door if it turns out to be wrong? Type-1 (one-way-door) decisions are hard or impossible to reverse and deserve slow, deliberate, senior-heavy process. Type-2 (two-way-door) decisions can be undone cheaply and should be delegated and decided fast, even with weak data.

The failure mode Bezos described directly is applying Type-1 rigor to Type-2 decisions: a company that requires the same committee review, financial modeling, and sign-off chain for a reversible pricing experiment as it does for an acquisition ends up slow, needlessly risk-averse, unwilling to experiment enough, and consequently worse at inventing. Most organizations don't lack judgment; they lack a fast, shared way to sort a decision into the right bucket before the debate starts.

What actually distinguishes the two types

The door test isn't about how scary or expensive a decision feels in the moment — a decision can feel high-stakes and still be a two-way door. What matters is cost and speed of reversal, not size:

DimensionType-1 (one-way door)Type-2 (two-way door)
ReversibilityHard, slow, or impossible to undoCheap and fast to undo
Who should decideSmall senior group, CPO/CEO involvedDelegate to the owning team
ProcessDeliberate, written memo, multiple perspectivesFast, informal, "disagree and commit"
Data barAct on ~70% of desired informationAct on far less; the trial is the data
ExampleSunsetting a product line, entering a new market, a pricing-model overhaulA feature flag rollout, a landing-page test, a support-process tweak
Cost of being slowUsually low relative to cost of being wrongHigh — delay itself is the main risk

Reading down that table, the practical move is to default every decision to Type-2 until someone can show it isn't — the bias in most companies runs the other way, treating anything unfamiliar as one-way simply because it's unfamiliar. A useful gut-check: name the smallest reversible experiment that would answer 80% of the question, and if one exists, you're in Type-2 territory even if the eventual full commitment is large.

Capital-allocation calls are the clearest Type-1 case at the CPO level — shutting down a business line or reallocating budget away from it is genuinely hard to reverse once teams have disbanded and customers have moved on, which is why a portfolio capital-allocation view matters before the door swings shut. Pricing changes, by contrast, are more Type-2 than most CPOs treat them: a poorly-received price increase can usually be walked back or grandfathered, worth remembering anytime a monetization and pricing strategy debate starts to feel more permanent than it actually is.

Calibrated Forecasting: Commit to a Number Before You Commit to the Decision

Calibrated forecasting means writing down what you expect to happen and how confident you are — as a number, not a feeling — before you act, then checking your track record over time. This habit is what separates a good decision-maker who happens to be right from one whose judgment is actually improving, because it's the only way to tell the difference between the two after the fact.

Psychologist Philip Tetlock's two-decade research program, culminating in the IARPA-funded Good Judgment Project, is the strongest evidence this works. Ordinary volunteers trained in a handful of calibration habits — breaking questions into components, updating in small increments, tracking their own accuracy — became "superforecasters" whose aggregate accuracy beat the average forecaster by a wide margin and reportedly outperformed intelligence-community analysts working with classified information. The skill wasn't domain expertise. It was the discipline of stating a probability and grading it later.

A related finding, popularized by Daniel Kahneman in Noise: A Flaw in Human Judgment (with Olivier Sibony and Cass Sunstein), adds the other half of the argument: two equally expert executives asked to judge the same situation will often give wildly different answers, and the same executive will disagree with their own past judgment on a different day. Kahneman calls this unwanted variability noise, distinct from bias — noise doesn't point in a consistent wrong direction, it just scatters. Writing a prediction down before deciding is one of the few reliable ways to catch it.

Why writing it down before you act is the whole mechanism

The order matters more than the number itself. A prediction made after the outcome is already known is worthless — hindsight quietly edits it — so the entire value of calibrated forecasting collapses if the "prediction" gets written after the fact, even by a day.

  • State the outcome you expect in concrete, checkable terms ("retention among enterprise accounts holds within 2 points of baseline," not "this should be fine").
  • Attach a confidence level — a rough percentage or a verbal band ("likely," "toss-up," "unlikely") is enough; precision theater isn't the point.
  • Timestamp it before the decision goes live, ideally in writing that a second person could read later without you explaining it.
  • Set a review date tied to when the outcome will actually be observable — not so soon that the signal is noise, not so late that you've moved on.

Author and former professional poker player Annie Duke names the trap this habit is built to avoid in Thinking in Bets: "resulting," or judging the quality of a decision purely by how the outcome turned out. A well-reasoned 70%-confidence call that fails 30% of the time is not a bad decision every time it fails — but only a logged prediction lets you tell a bad decision from a bad break.

The Decision Journal: A Worked Example

A decision journal entry captures four things at the moment of the call — the decision, the door type, the prediction with a confidence level, and the assumptions it rests on — so the later review has something concrete to grade against instead of a reconstructed memory. Below is one worked example, end to end.

The situation: a mid-market SaaS company's CPO is deciding whether to raise prices 15% on the core plan, the first increase in three years, ahead of a competitor's rumored feature-parity release.

Step 1 — classify the door. This is closer to Type-2 than it feels: existing customers can be grandfathered, and the increase can be rolled back or slowed for new cohorts within a quarter if churn spikes. Verdict: Type-2, act at roughly 70% confidence, don't over-process.

Step 2 — write the prediction and confidence, dated before launch:

"I predict enterprise-tier churn rises no more than 1.5 points over the following two quarters, and net revenue from the price increase is positive by month four despite some churn. Confidence: 65%. Key risk: the competitor's feature-parity release lands before our renewal cycle completes." — logged the day the decision was approved.

Step 3 — log the assumptions the prediction depends on. This is the step most executives skip, and it's the one that makes the eventual review useful rather than a vague "were we right?":

  1. Renewal-cycle timing gives most enterprise accounts a full quarter's notice before the increase hits their invoice.
  2. The competitor's feature-parity release is at least two quarters out, based on their public roadmap signals.
  3. Price sensitivity is concentrated in the smallest-seat accounts, not the accounts driving most revenue — worth checking against a jobs-to-be-done read of what those accounts actually hired the product to do, since a price-driven job gets more elastic than a mission-critical one.
  4. Support and success teams can absorb a modest uptick in downgrade requests without a corresponding drop in customer-journey satisfaction at the renewal touchpoint.

Step 4 — review on the date set, not whenever it's convenient. At the two-quarter mark, the CPO checks the actual churn number and revenue delta against the prediction, and — separately — checks each assumption individually rather than only the headline result:

ElementPredictedActualAssumption held?
Enterprise churn≤ 1.5 points1.1 points—
Net revenue positive by month 4YesYes, by month 3—
Renewal-cycle noticeFull quarterFull quarterHeld
Competitor release timing2+ quarters outShipped in month 5Did not hold
Price sensitivity concentrationSmallest-seat accountsConfirmedHeld

The outcome looks like a clean win on the surface. The assumption-level review tells a more useful story: the competitor shipped faster than expected, and the plan only survived that miss because churn was already running better than predicted elsewhere. That's the actionable insight — not "the price increase worked," but "our competitive-timing assumptions run optimistic, and the next decision that depends on a rival's roadmap should discount it further." That distinction is the entire point of separating outcome from process, and it's invisible if you only ever ask whether the number went up.

Building the Habit: From One Good Call to an Institutional Skill

A single well-logged decision teaches you almost nothing on its own — calibration is a property of a track record, not one entry, so the real payoff shows up only after a CPO and their team have accumulated a few dozen logged predictions to look back across. The habit has to survive past the decision that prompted it, or it's just a diary entry.

Harvard Business School's Amy Edmondson, in Right Kind of Wrong, makes the culture case for why this is hard to sustain: reviewing a decision honestly requires distinguishing an "intelligent failure" — a good bet that didn't pay off — from a preventable mistake, and most organizations lack the psychological safety to make that distinction out loud without it turning into blame. A CPO who reviews their own logged decisions in front of the team, including the ones that missed, sets the norm that a wrong call reviewed honestly is worth more than a right call nobody examined.

Three structural habits keep the practice alive past the first few entries:

  • Review on a fixed cadence, not only when a decision has clearly gone wrong — a quarterly pass across the log catches quiet drift in your calibration before it compounds.
  • Track a calibration curve, not just a win rate. Bucket your logged predictions by confidence band (50-60%, 60-70%, and so on) and check whether the ones you called "70% confident" actually landed around 70% of the time — a forecaster who's right 90% of the time on their "70%" calls isn't well-calibrated, they're under-confident.
  • Separate the review from the retro. A team retro asks what to do differently next time; a calibration review asks a narrower question — was my stated confidence justified by what I actually knew at the time — and conflating the two lets teams skip the harder one.

This discipline is one thread in a broader executive skill set — sequencing it alongside capital allocation, monetization, and org design is the subject of the full CPO playbook, and it shows up early for a reason: a new CPO's first 90 days are almost entirely Type-1 decisions made on thin information, which makes them the highest-value place to start a decision log rather than waiting for a "big enough" call to justify one.

Where Prodinja fits

This is the territory Prodinja's Leadership Suite is built for. The Decision Dojo walks you through named high-stakes scenarios — the pricing call, the sunset decision, the market-entry bet — so you can practice the Type-1/Type-2 sort and the prediction-before-action habit before a live decision forces the pace. The Decision Journal is designed to capture your predicted outcome and confidence at the moment you commit, then resurface it on your review date so calibrating your judgment over time is a habit the tool supports rather than a spreadsheet you have to maintain yourself.

Neither replaces the reasoning — Bezos's door test and Tetlock's calibration discipline work with a notebook and a calendar reminder. What structured tooling adds is the part most executives quietly drop: the consistent timestamped log and the scheduled review that turns isolated good calls into a measurable, improving skill.

Key Takeaways

  • Reframe the executive question from "what data are we missing" to "have I classified this decision correctly and reasoned well given what I have" — the data gap at the top is structural, not a research failure.
  • Sort every high-stakes call as Type-1 (hard to reverse) or Type-2 (cheap to reverse) before debating its merits; most decisions default to Type-2 more than organizational habit assumes.
  • Write your prediction and confidence level before you act, not after — a post-hoc prediction is worthless because hindsight silently edits it.
  • Log the assumptions underneath a decision, not just the headline call — reviewing assumption-by-assumption surfaces which parts of your reasoning to trust next time, even when the overall outcome looks fine.
  • Separate decision quality from outcome quality. A good decision can still lose, and a lucky one can still win — Annie Duke's "resulting" trap conflates the two, and only a logged prediction lets you tell them apart.
  • Review on a fixed cadence and track a calibration curve across confidence bands, not just a running win rate, to see whether your "70% confident" calls actually land 70% of the time.
  • Build the practice into a system, not a memory — whether that's a shared log, a recurring calendar review, or a tool like Prodinja's Decision Journal built for exactly this loop.

Frequently Asked Questions

What is the Type-1 vs Type-2 decision framework?

It's Jeff Bezos's model for sorting decisions by reversibility: Type-1 (one-way-door) decisions are hard or costly to undo and warrant slow, senior, deliberate process, while Type-2 (two-way-door) decisions are cheap to reverse and should be made fast and delegated. The framework's core value is preventing organizations from applying Type-1 rigor to Type-2 calls, which is what usually causes executive-level decision paralysis.

How do you make good decisions with incomplete data?

Classify the decision's reversibility first, then act on roughly 70% of the information you'd ideally want rather than waiting for more — Bezos's own benchmark — while writing down a specific, checkable prediction and confidence level before committing. Calibrated forecasting research from Philip Tetlock's Good Judgment Project shows this habit, not raw expertise, is what improves judgment over time.

What is a decision journal and how do you start one?

A decision journal is a written log capturing, at the moment of each significant decision, what you decided, why, your predicted outcome, your confidence level, and the assumptions the prediction depends on. Start with one entry per genuinely high-stakes call, set a review date tied to when the outcome becomes observable, and review the assumptions individually rather than only the headline result.

How is calibrated judgment different from just being decisive?

Decisiveness is about speed; calibration is about the accuracy of your stated confidence over a track record of decisions. A decisive executive who is consistently overconfident will keep making fast calls that miss more often than their stated confidence implies — calibration training, per Kahneman's and Tetlock's research, corrects that gap by grading predictions against outcomes rather than rewarding speed alone.

Can calibrated decision-making be taught, or is it innate judgment?

It can be taught — Tetlock's Good Judgment Project trained ordinary volunteers in specific habits (breaking questions down, updating in small increments, tracking accuracy) and measured them outperforming untrained forecasters and, in aggregate, professional intelligence analysts. The mechanism is deliberate practice against real feedback, the same structure Prodinja's Decision Dojo scenarios and Decision Journal are designed to support for executive-level calls specifically.