Vanity Metrics in Disguise: Auditing Key Results That Only Go Up

A key result is vanity if it can only go up and its movement never tells you what to do next. Total signups, total downloads, and total content published all climb by definition, so hitting them proves activity happened — not that the business actually improved this quarter. Swap them for rates, ratios, or cohort measures that can genuinely rise or fall.

Quick Answer: Run every key result through two questions: can the number go down, and does a change in it tell you what to do next? Fail either one and it's vanity. Rewrite cumulative counts (total signups) as rates with a real baseline (weekly activation rate, up from 22% this quarter) — a target that can actually fail is the only kind worth committing to.

The Two-Question Vanity Test

A metric earns a place in an OKR when it can go down and when a move in either direction tells the team something to do. Fail either test and you've written a key result you can hit every single quarter without anything real changing — which is exactly why so many teams keep writing them.

Eric Ries drew the original line in The Lean Startup (2011): a vanity metric looks good on a slide and moves in one comforting direction, while an actionable metric ties a specific, repeatable action to a specific, expected change in the number. His example was total registered users versus a cohort-based conversion rate — the same distinction that shows up in OKRs a decade later, just wearing a different name.

Run any draft key result through this pair of questions:

1. Can it go down? If the metric is a running sum — total users, total revenue booked to date, total tickets closed — the answer is structurally no. It only ever adds.
2. Does a change imply a decision? If the number moves and nobody on the team would do anything differently, it isn't actionable — it's a scoreboard.

A key result that fails question one almost always fails question two as well, because a number that can't fall can't signal a problem either. That's not a coincidence — it's the same underlying flaw showing up twice.

Andy Grove, whose Intel management practice is where OKRs originated, insisted a key result had to be unambiguous and verifiable — "there is no subjective judgment in it," as John Doerr later paraphrased him in Measure What Matters (2018). A number that only ever rises can still be "verifiable" in the narrow sense, but it verifies nothing except that time passed.

If you want the fuller mechanics of writing key results before you get to auditing them, the complete guide to advanced OKRs covers format, cadence, and scoring end to end. This piece assumes you already have a KR and are checking whether it deserves to survive.

Why Cumulative Counts Always Climb and Rates Don't

Cumulative counts climb because they're structurally addition-only: every period sums onto the last, so the running total cannot fall unless someone manually subtracts churn, which almost no dashboard does automatically. Rates and ratios divide one changing number by another, so a shrinking numerator or a growing denominator shows up the moment it happens.

This is the mechanical reason vanity metrics feel so safe to commit to. Total signups this quarter is last quarter's total signups plus whatever came in — it is mathematically impossible for it to end the quarter lower than it started, short of the company deleting accounts. Compare that to weekly activation rate, which is:

activations this week ÷ signups this week

Both the numerator and the denominator reset every period, so the ratio can rise, fall, or flatline depending on what actually happened to the cohort that just arrived — not what happened to every cohort that ever arrived.

This is really the outcome-vs-output distinction showing up in numeric form. A cumulative count is almost always counting output — units produced, users acquired, tickets closed — while a rate is far more likely to be measuring an outcome, because it has to account for a denominator the team doesn't fully control.

Google's internal OKR guidance, published through its re:Work initiative, makes the same point in plain language: key results should read as outcomes a reasonable person could disagree you'd hit, not a count of activity a team can pad by doing more of the same thing.

Three Vanity Key Results, Rewritten as Actionable Ones

The fastest way to internalize the test is to see it applied. Below are three key results that show up constantly in planning docs, paired with a rewrite that keeps the same underlying intent but makes the number capable of failing.

A few things worth noticing across all three rewrites:

• The denominator is the whole point. Weekly activation rate forces you to look at signups and activations together; a team can't quietly inflate the numerator without the ratio exposing it.
• Each rewrite names a real baseline. "From 22% to 32%" is falsifiable in a way "reach 50,000" never is — you can miss 32%, and missing it means something.
• None of the rewrites ban the original count from being tracked. Total signups is still a fine input metric to watch on a dashboard. It just doesn't belong as a key result, because hitting it proves nothing about whether the product got better.

This distinction matters because raw counts are exactly the kind of target that invites Goodhart's Law behavior: once a number becomes the goal, teams find the cheapest way to move it rather than the real one. In practice that cheapest way usually looks like:

• A spike in low-quality signups bought from a paid campaign
• A flurry of thin blog posts padding out the content count
• Tickets closed by asking the customer to simply reopen a new one

The piece on killing goal theater in the quarterly cycle goes deeper on how teams end up performing progress instead of making it — vanity KRs are usually the mechanism that makes the performance possible.

Cohort and Ratio Framing: Building Key Results That Can Actually Fail

Two techniques do most of the work of converting a vanity count into something actionable: cohort framing, which measures a defined group through the same window instead of blending it with everyone who came before, and ratio framing, which expresses the target as one changing number over another. Used together they make a key result that can genuinely miss.

Cohort framing

A blended, all-time metric hides recent damage behind years of accumulated history. If your product has 500,000 lifetime users and this month's cohort activates at half the usual rate, the all-time activation percentage barely moves — the new cohort is a rounding error against the old one.

Cohort framing fixes this by tracking each period's arriving group through an identical window:

• Week-1 cohort: of everyone who signed up this week, what share completed the core action within 7 days?
• Month-1 retention cohort: of everyone who signed up this month, what share is still active 30 days later?
• Release cohort: of everyone who saw the new onboarding flow, what share activated, compared with the cohort that saw the old one?

Because each cohort is measured on its own terms, a real regression in this week's experience shows up in this week's number — it can't hide behind twelve quarters of accumulated users. This is the same logic behind mapping a customer journey: a journey stage only means something when you track a defined group moving through it, not a blended average across everyone who ever touched the product at any stage.

Ratio framing

Ratio framing is the more general version of the same idea: express the key result as numerator ÷ denominator rather than a raw count, and make sure both sides are free to move.

• Instead of "total qualified leads," use qualified leads ÷ total leads (lead quality rate).
• Instead of "total feature adoptions," use weekly active users of feature X ÷ weekly active users overall (adoption share).
• Instead of "total NPS responses collected," use the actual NPS score for the period — a score, not a response count.

Ratio framing also connects naturally to JTBD-style thinking. A customer's job was never "sign up" or "download the app" — those are vanity events a product tracks because they're easy to log, not because they're what the customer hired the product to do.

An activation rate defined around the customer's actual core action gets much closer to measuring whether the job got done. That's exactly the kind of specificity the complete guide to jobs-to-be-done pushes teams toward when defining what "progress" means for a real customer.

Auditing Your Own OKRs Before You Commit

An OKR audit is a five-step pass through your current key results, applying the two-question test to each one and rewriting anything that fails before the quarter starts, not after it's already been missed or trivially hit. Do this in planning, not in a retro — a vanity KR discovered in week eleven has already wasted the quarter.

1. List every key result exactly as written. Not the intent behind it — the literal sentence someone would score against.
2. Apply the two-question test to each one. Can it go down? Does a change imply a decision? Flag anything that fails either.
3. Find the missing denominator. Almost every vanity KR is a numerator without one — total signups is missing "out of how many visited"; total tickets closed is missing "out of how many were opened, and how well."
4. Rewrite as a rate or ratio and pull a real baseline before committing. A target with no baseline is a guess dressed as a goal — go get last quarter's actual number first.
5. Cascade the rewrite without letting subteams invent their own vanity counts underneath it. A team-level KR that rolls up into a company rate should usually still be a rate or a clearly bounded contribution to one — see cascading OKRs without the waterfall for how to do that without every layer restating the same number in smaller boxes.

Where the audit gets harder than it sounds

Leadership sometimes wants the big round number — "50,000 signups" is a better headline in a board deck than "activation rate up 8 points" — and that pressure is exactly how vanity KRs survive the audit anyway. Naming the tradeoff explicitly (you can report the count publicly, it just can't be the thing the team is scored against) usually resolves it faster than arguing about the metric itself.

Prodinja's Spec Studio has a Success Metrics section built around this same discipline for individual features: it asks for a primary metric, a success threshold expressed as an actual number, a time window, and a guardrail metric that must not regress.

A guardrail only makes sense for a number capable of regressing — try entering a cumulative total there and the fields start to feel wrong, because a running count has no floor it could ever cross. Rates and ratios with an honest baseline fit that shape naturally, which is the nudge the section is designed to give rather than a verdict it hands down.

Key Takeaways

• A key result is vanity if it fails either of two tests: can it go down, and does a change in it imply a decision. Both failures usually travel together.
• Cumulative counts are monotonic by construction — total signups, total downloads, and total tickets closed can't fall without manual intervention, so hitting them proves activity, not progress.
• Rates and ratios divide two changing numbers, which is what lets them genuinely rise or fall based on what happened in the period being measured.
• Cohort framing stops a blended, all-time metric from hiding a real recent regression behind years of accumulated history.
• Ratio framing (numerator ÷ denominator) is the general technique behind most actionable-metric rewrites, from activation rate to lead quality rate.
• Every rewrite needs a real baseline pulled before the quarter starts — a target with no baseline is a guess wearing a goal's clothes.
• A vanity count can still be tracked as an input metric on a dashboard; it just shouldn't be the key result a team is scored against.

Frequently Asked Questions

What's the difference between a vanity metric and an actionable metric?

A vanity metric moves in one direction (usually up) regardless of what the team does, so it can't tell anyone what to change. An actionable metric ties a specific action to an expected, falsifiable change — it can rise, fall, or hold flat depending on real behavior, which is what makes it useful for a decision.

Can a cumulative metric ever be a valid key result?

Rarely, and usually only when there's a hard ceiling that makes the count meaningfully bounded — "close 100% of a fixed 40-account enterprise migration list" behaves more like a checklist than an open-ended vanity count. For anything with no natural ceiling (signups, downloads, content pieces), convert it to a rate before committing to it.

How do I find the baseline for a rate I've never tracked before?

Pull the raw numerator and denominator for the last one to three completed periods from existing data — signups and activations, leads and qualified leads — and calculate the rate retroactively. If the instrumentation doesn't exist yet, the first quarter's key result should be "instrument and baseline the rate," not a guessed target.

Why do teams keep writing vanity key results if everyone agrees they're weak?

Because they're comfortable: they're nearly guaranteed to be hit, they're easy to report upward without nuance, and they don't require anyone to expose a real baseline they might miss. The incentive runs toward vanity KRs unless someone explicitly audits for them every planning cycle — which is also why they're a recurring symptom of goal theater in the quarterly cycle.

Is total revenue always a vanity metric?

Total revenue booked to date is cumulative and vanity for the same structural reason as total signups. Recurring-revenue rates that can genuinely fall — net revenue retention, quarter-over-quarter growth rate, gross margin — are the actionable versions worth using as key results instead.