Tracking Fatigue: Why Measuring Everything Means Understanding Nothing

A thousand tracked events is not rigor — it's noise. Most dashboards answer questions nobody is asking, while the five signals a team actually needs to decide something sit buried under redundant, half-named, high-cardinality clutter nobody has the discipline to prune.

Quick Answer: Tracking fatigue happens when event volume outpaces decision volume. Fix it by requiring every metric to pass the "if this moved, what would we do?" test, running a quarterly event audit, and letting a bet-and-threshold structure — not a wishlist — decide what gets instrumented.

Why More Metrics Make Teams Understand Less

More metrics reduce understanding because the cost of finding a signal rises faster than the number of signals does. Every additional event adds search cost to every future analysis, so a team with 900 events answers questions slower than a team with 40, even though it "knows more" on paper.

This isn't a discipline problem you fix by hiring a stricter analyst. It's a structural one: dashboards are cheap to create and expensive to delete, so they accumulate monotonically unless something forces a deprecation step. Miller's classic finding on the limits of working memory — that people reliably track only about seven, plus or minus two, chunks of information at once — is about short-term recall, not dashboard design, but the underlying constraint transfers: a human being still has to hold a handful of signals in mind to reason about a product decision, no matter how many are logged.

Event tracking without a retirement mechanism behaves like unbounded technical debt. Nobody sets out to build a 1,000-event schema. It happens one well-intentioned button_clicked_v2 at a time, and each addition looks individually harmless because nobody audits the marginal cost of the 743rd event against the value of the first 20. A tracking plan built before code is the standard fix on the way in; the audit ritual below is the fix for what already accumulated.

The Volume-vs-Decisions Gap

Tracking fatigue is measurable as a ratio: events instrumented divided by decisions those events actually inform. Healthy teams keep that ratio close to 1:1 per product area; bloated ones drift toward 50:1 or worse without anyone noticing, because nobody is counting the denominator.

The table's point in one sentence: fatigue isn't about absolute event count, it's about the ratio of instrumentation to actual usage — and that ratio only gets worse without an active pruning habit.

The "If This Metric Moved, What Would We Do?" Test

A metric earns a place in the catalog only if you can name the specific action a change in its value would trigger — if no answer exists, the metric is decoration, not decision support. This single question is the cheapest filter available and it should run before instrumentation, not after.

Apply the test as a three-part check whenever someone proposes a new event or metric:

1. Name the decision. What choice does this number inform — ship, kill, reprioritize, escalate?
2. Name the threshold. At what value does the decision actually change? "We'd revisit onboarding copy if completion drops below 60%" is a real threshold; "we'd want to know" is not.
3. Name the owner. Who is accountable for acting when the threshold trips? A metric with no owner is a metric nobody will ever act on.

Metrics that fail all three are usually vanity or curiosity metrics — interesting to look at, connected to no lever anyone will pull. That doesn't make them worthless for research, but it does mean they don't belong in the core operating dashboard, where every extra tile is a tax on the metrics that do matter.

Andy Grove's original formulation in High Output Management — pairing every output metric with a counter-metric that prevents gaming — is a useful companion filter here. If a metric has no natural counter-metric, that's often a second sign it was added reflexively rather than deliberately.

A Worked Example

Consider page_scroll_depth_25_50_75_100 fired as four separate events per page view. Ask the test: what decision changes if scroll depth at 75% drops from 40% to 30%? For most teams, none — it's logged because scroll-depth tracking is a common analytics-tool default, not because a decision hangs on it.

Compare that to checkout_abandoned_at_step. The decision is obvious (which step in the flow needs redesign), the threshold is nameable (a step with abandonment 15+ points above the flow average), and the owner is clear (the checkout squad's PM). One event survives the test; the other is a candidate for pruning.

Building a Question-First, Deprecation-Friendly Catalog

A question-first catalog starts from the decisions a team needs to make and works backward to the minimum events required, rather than starting from "what can we technically capture" and hoping insight falls out. Deprecation-friendliness means every entry has a built-in expiration review, not just a creation date.

Structure the catalog around four fields per metric, borrowing the discipline of an event naming taxonomy and convention so the catalog itself stays legible as it grows:

• Question — the specific thing this metric answers (not a topic area, an actual question)
• Decision — what changes when the answer moves
• Owner — who's accountable for the metric staying relevant
• Review date — a forced re-justification, typically quarterly

A metric with no review date is a metric that will outlive its usefulness by default — silence is the natural state of a dashboard, not evidence it's still needed.

This is also where event property design and schema decisions matter as much as the event name itself. A well-named event with ten loosely-typed properties can still generate fatigue at the property level, because analysts have to guess which of those ten actually matter for any given question.

What to Cut First

When triaging an existing bloated catalog, cut in this order, because each category has the worst ratio of maintenance cost to decision value:

1. Duplicate-intent events — three different events all approximating "user engaged," none owned by the same team
2. Orphaned events — no owner, no dashboard reference, no query against them in the last two quarters
3. Unbounded free-text properties — the biggest tooling-cost driver, covered below
4. One-off launch metrics — instrumented for a single experiment, never retired after it shipped

Cutting in this order clears the highest-cost, lowest-value events first, which is usually enough to make the remaining catalog feel navigable again even before the harder judgment calls.

The Event-Audit Ritual

A recurring, scheduled event audit is the only mechanism that reliably keeps a tracking plan from re-bloating, because deprecation never happens by default — it has to be manufactured as a ritual with a date on the calendar. Treat it like a recurring maintenance job, not a one-time cleanup project.

Run it quarterly, with a fixed agenda:

1. Pull query logs. Which events were actually queried, in a dashboard or ad hoc, in the last 90 days? Most analytics warehouses can report this directly.
2. Flag zero-query events. Anything with zero queries in 90 days goes on the chopping block by default; the owner must actively justify keeping it.
3. Re-run the "what would we do" test on everything that survives step 2 — a metric can be queried occasionally out of habit without any decision actually depending on it.
4. Check cardinality and volume against tooling cost (next section) for anything flagged as expensive.
5. Publish a deprecation list with a sunset date, typically 30-60 days out, so downstream dashboard owners have time to react.
6. Retire, don't just ignore. Actually stop firing the event; a "deprecated but still logging" event still shows up on the bill and in autocomplete.

Anchoring this ritual to something durable — like the questions in a customer journey or a set of jobs to be done the product serves — keeps the audit from becoming purely mechanical. The point isn't to hit a smaller number; it's to keep every remaining event tied to a real question about how customers experience the product.

Who Should Own the Ritual

Assign a single accountable owner for the audit itself — usually a PM or analytics lead — even though individual metric decisions are distributed to feature owners. Without one accountable person scheduling and chasing the ritual, "we should really clean this up" becomes a permanent backlog item that never gets prioritized against feature work.

The Cost of High-Cardinality Events on Tooling Bills

High-cardinality event properties — raw user IDs, unbounded free text, timestamps stored as string properties — inflate analytics and observability bills disproportionately to the insight they produce, because most usage-based pricing scales with the number of distinct property values, not the number of events. A property with millions of unique values can multiply the effective cost of a single event several times over.

Cardinality drives cost in a few concrete ways:

• Distinct-value explosion. Many tools index or bucket by property value; an unbounded field (a raw search string, a full URL with query params) can generate orders of magnitude more distinct combinations than a bounded enum would.
• Storage and indexing overhead. Time-series and event-analytics backends often charge based on cardinality of the metric-plus-tags combination, not just row count — this is a well-documented cost driver in observability tooling generally, not unique to any one vendor.
• Query slowness that compounds the fatigue problem. High-cardinality data is slower to aggregate, which pushes analysts toward pre-built dashboards instead of ad hoc questions — reinforcing the exact rigidity that caused the bloat in the first place.

Practical fix: bucket free text into a bounded taxonomy wherever possible, sample high-volume low-value properties instead of capturing every instance, and never put a raw unique identifier into a property meant for grouping or filtering — put it in a lookup table instead. A well-scoped event property schema is the design-time version of this fix; the audit ritual is the maintenance-time version.

A Quick Cardinality Gut-Check

Before shipping a new property, estimate its likely distinct-value count against its actual analytical use:

The practical takeaway: any property whose cardinality approaches the number of users or sessions is a cost and performance liability disguised as "extra detail," and it rarely earns its keep against the "what would we do" test either.

Where a Bet-and-Threshold Structure Forces Pruning

This whole approach is easier to operationalize when the underlying prioritization system requires every proposed metric to trace back to a specific bet and a decision threshold, rather than letting metrics accumulate as an unstructured wishlist attached to whatever feature shipped most recently.

That's not a claim that a tool alone solves tracking fatigue — the audit ritual and the question-first catalog above are the actual discipline. But a prioritization process that structurally requires a threshold per bet makes the discipline much easier to sustain, because the "why does this metric exist" question gets asked once, at the point of highest leverage, rather than never.

Key Takeaways

• Tracking fatigue is a ratio problem — events instrumented versus decisions actually informed — not a raw count problem, so the fix is pruning against decisions, not hitting an arbitrary event-count target.
• The "if this metric moved, what would we do?" test is the cheapest and most reliable filter for both new metrics and existing ones; anything that fails on decision, threshold, or owner is a pruning candidate.
• A question-first, deprecation-friendly catalog — question, decision, owner, review date — keeps a tracking plan legible as it scales, instead of accumulating indefinitely.
• A quarterly event-audit ritual with query-log review, a re-run "what would we do" test, and a published deprecation list is the only mechanism that reliably counters natural dashboard accumulation.
• High-cardinality properties are a hidden tooling-cost driver, often scaling analytics bills faster than event volume itself — bound free text, sample where possible, and keep raw IDs out of grouping properties.
• A bet-and-threshold prioritization structure gives every metric a reason to exist upfront, which is a far more sustainable filter than trying to clean up bloat after the fact.