How to Measure Whether Your Data Products Are Actually Used

Measuring whether a data product is actually used means tracking four distinct layers—reach, active use, trust, and decision impact—not just query volume. A table queried twice a quarter can be a bigger win than a dashboard opened daily, if that rare query drives a board decision and the daily dashboard drives none.

Quick answer: Query counts measure exposure, not value. Track reach (who has access), active use (who returns without being told to), trust (do people believe the numbers), and decision impact (did it change an action)—then weight each data product by how consequential its use is, not how often it gets opened.

Why Query Counts and Dashboard Views Don't Prove Adoption

Query counts and dashboard views measure exposure, not value—they conflate curiosity with commitment and say nothing about whether a decision changed because of the data. A metric that only counts visits rewards the dashboard people check out of habit and quietly punishes the two-person, quarterly-use table that powers a pricing model.

This is the same trap Eric Ries named vanity metrics in product management: numbers that go up and to the right without telling you whether anything real happened. Query count, unique viewers, and dashboard opens are the data-product equivalent of "page views." They correlate weakly, if at all, with whether a decision-maker trusted the number enough to act on it.

Gartner has warned for years that a majority of data and analytics investments fail to deliver their promised business value—not because the data is technically wrong, but because it never gets adopted into how decisions actually get made. That's an adoption failure hiding behind a technically successful pipeline. If your only dashboard is SELECT COUNT(*) FROM query_logs, you can't tell the difference between the two.

Treating a data product like a real product—with a user, a job, and a decision it's supposed to unblock—is the mindset shift behind treating data products as if they have real users. Once you adopt that lens, query volume stops being the finish line and starts being one weak proxy among several.

The three failure modes query counts hide

• False positive: A dashboard gets opened daily because it's the default landing tab, not because anyone acts on it.
• False negative: A critical table is queried rarely because it feeds one high-stakes monthly close process, not because it's unused.
• Silent churn: A user tries a data product once, doesn't trust the number, and never comes back—but that single visit still counts as "usage" forever.

Any of these three can make a mediocre data product look like a hit, or a mission-critical one look like a flop, if query count is the only number in the room.

DAMA International's Data Management Body of Knowledge (DMBOK) makes a related point when it defines data governance maturity: it treats "fit for use" as a property judged by the consumer, not the producer. A data PM who only reports usage volume is implicitly letting the pipeline grade its own homework, when the whole point of the governance discipline DMBOK codifies is that the consumer's judgment is the one that counts.

The Four-Layer Adoption Model for Data Products

A layered model separates reach (who could use it), active use (who does), trust (do they believe it), and decision impact (did it change anything)—each layer catches a failure the previous one misses. A data product can pass reach and active use while failing trust and decision impact, and that combination is the most common way "high usage" masks a low-value product.

Each layer answers a different question a query-count dashboard can't:

The table above is roughly ordered by how hard each layer is to measure and how often teams skip it—most data teams instrument reach and active use well since it's just event logging, and almost never instrument trust or decision impact, because those live in Slack threads, meeting notes, and a stakeholder's head, not in a query log.

That gap matters because data quality is what earns the trust layer, not the other way around. A data product can have perfect reach and healthy active-use numbers and still fail if one bad number a year ago made a VP quietly stop trusting it—that's churn a query log will never show you, because the VP's analyst still runs the query for them.

Mapping the Adoption Funnel for Internal Data Products

An adoption funnel for internal data products runs through six stages—awareness, access, first use, repeat use, trust, and decision impact—and the honest failure point for most internal tools sits between repeat use and trust, not between awareness and access. Treat it like any product funnel: measure the drop-off at each gate, not just the top and bottom.

Building this funnel for a real data product is a straightforward, repeatable exercise:

1. Define the intended audience for the product—not "anyone with a warehouse login," but the specific roles the product was built to serve.
2. Instrument access and first use with basic event logging, which most warehouses and BI tools already emit.
3. Separate repeat use from one-time curiosity by tracking distinct active weeks per user, not lifetime query count.
4. Add a lightweight trust check at the point of use, such as an in-tool "did you use this in a decision?" prompt or a quarterly survey.
5. Trace decision impact backward from known decisions—a pricing change, a board metric, a retired spreadsheet—to the data product that fed them, rather than forward from the query log.

This is the same discipline behind mapping a customer journey with an emotion curve for an external product: stage-by-stage instrumentation, honest drop-off tracking, and a refusal to let one aggregate number stand in for the whole path. Internal data consumers deserve the same rigor external customers get.

A common objection is that internal users don't behave like a funnel—they're a captive audience who can't just churn to a competitor. That's true, but it makes the funnel more diagnostic, not less: when a captive user still abandons a data product for a spreadsheet workaround, the drop-off is a purer signal of a real problem, because there was no competitive alternative pulling them away.

Surveying for Trust and Job Satisfaction, Not Just Usage

A short, recurring survey captures two things a query log structurally cannot: whether a data consumer trusts the number enough to act on it, and how satisfied they are with the job the data product is supposed to help them do. Pair those two questions and you get a cleaner adoption signal than any usage dashboard.

Borrow the two-question structure from Tony Ulwick's Jobs-to-be-Done research: ask how important a specific job is, then how satisfied the person is with their current ability to do it. The gap between the two, not the raw satisfaction score, is where the opportunity lives. That's the same logic behind the Jobs-to-be-Done framework applied to any product, internal or external.

A minimal quarterly survey for a data product's core audience needs only a handful of questions:

• "How important is [decision/task] to your role?" (1-10)
• "How satisfied are you with your current ability to do that, using this data product?" (1-10)
• "How much do you trust the numbers in this product enough to cite them without double-checking?" (1-5)
• "In the last month, did this data product change a decision, or did you just check it?" (open text or forced choice)
• One open-ended prompt: "What would make you trust this data product more?"

The math behind the gap is simple and borrowed directly from Ulwick's opportunity scoring formula: Opportunity = Importance + max(Importance − Satisfaction, 0). A job scored as highly important but poorly satisfied produces a high opportunity score—exactly the data products worth investing in next, regardless of how many queries they logged last month.

Fred Reichheld's research behind the Net Promoter Score at Bain & Company established a related principle worth borrowing here: a single, well-chosen relationship question, asked consistently over time, often predicts behavior better than a pile of activity metrics. A trust score tracked quarterly, even a crude one, tends to move before churn shows up in the usage logs, not after.

And trust doesn't stay static once you build it. A metrics layer the whole company can actually trust is what keeps survey answers from drifting downward every time two teams report a different number for the same metric—inconsistent definitions are one of the fastest ways to erode the trust layer the survey is trying to measure.

Finding the Real Signal: A Quiet Table Can Beat a Busy Dashboard

A rarely-queried table can be a bigger success than a heavily-used dashboard when its query volume is low but its decision weight is high—one feeds a monthly close, a regulatory filing, or a pricing model, while the other gets glanced at out of habit. Query count and decision impact are simply different axes, and a data product can score high on one and low on the other.

The honest takeaway from this table is that query count and decision impact need to be reported side by side, never as substitutes for each other. A data PM accountable for outcomes across a portfolio of tables, dashboards, and pipelines needs both numbers per product, not one number averaged across the whole catalog—which is exactly the accountability gap a broader guide to the data PM role has to reckon with when defining what "success" means for this job.

Where this shows up inside Prodinja

This importance-versus-satisfaction gap is the exact measurement Prodinja's Customer Jobs tool is designed to support: you log the job a data consumer is trying to do, score its importance and their current satisfaction, and the tool computes the Ulwick-style opportunity score for you. For a data PM, that means a finance analyst's rarely-asked "can I trust this number for the close" job can outscore a heavily-used dashboard's "can I see this at a glance" job, because the gap between importance and satisfaction—not the click count—is what the tool is built to surface.

Key Takeaways

• Query counts and dashboard views are vanity metrics for data products—they measure exposure, not whether anyone trusted or acted on the number.
• Use a four-layer model—reach, active use, trust, decision impact—so a data product can't look successful just because it clears the easiest layer to measure.
• Build an explicit adoption funnel (awareness, access, first use, repeat use, trust, decision impact) and instrument the drop-off between stages, not just the totals.
• A short, recurring survey pairing importance and satisfaction using Ulwick-style opportunity scoring captures trust and job-fit that no query log ever will.
• A rarely-queried table can outrank a busy dashboard the moment you weight by decision consequence instead of raw traffic.
• Report query volume and decision impact side by side, never as substitutes, especially across a portfolio with very different usage patterns.
• Trust is fragile and cumulative: one bad number can silently erode adoption long before it shows up as a drop in the usage logs.