Precision, Recall, and F1 Without the Math Degree

Precision measures how many of your model's positive calls were actually correct; recall measures how many of the real positives it actually caught. F1 averages the two into one tidy number — which is exactly the problem, because a spam filter and a fraud detector need that trade-off pushed in opposite directions, and no single score can tell you which way to push.

Quick answer: Precision = of everything you flagged, how much was real. Recall = of everything real, how much you caught. Favor precision when a false alarm is the expensive mistake; favor recall when a miss is. F1 blends them evenly and hides which one your product actually needs.

These three numbers are the numeric core of any classification eval — our complete guide to AI evals covers how they sit inside a full evaluation practice. The arithmetic is genuinely the easy part. The judgment call — which kind of wrong you're willing to be more often — is the part most teams skip, and it's the part this article is actually about.

The Confusion Matrix in Plain Product Terms

A confusion matrix is a 2x2 scorecard for any yes/no prediction. It sorts every case into one of four buckets — true positive, false positive, true negative, false negative — based on what your model predicted versus what actually happened. Every metric in this article is just arithmetic performed on those four counts.

Before you can fill in that scorecard, you need examples where you already know the right answer. That's what our guide to building your first golden dataset with 50 examples calls the minimum viable ground truth. Without labeled examples, precision and recall aren't measurements — they're vibes wearing a decimal point.

Read the table left to right: TP and TN are the model behaving correctly; FP and FN are the two distinct ways it can be wrong. Precision, recall, and F1 are just different ways of weighting how much each type of "wrong" should count against the model.

One number to distrust immediately: accuracy, or (TP + TN) divided by everything. It looks reassuring on imbalanced data even when a model is nearly useless — if only 1% of transactions are fraud, a model that flags nothing at all still scores 99% accuracy while catching zero fraud. Precision and recall exist specifically because accuracy can't see that failure.

False Positives vs. False Negatives: Annoying vs. Dangerous

A false positive and a false negative are never equally bad — the cost depends entirely on what the user loses in each direction. A spam filter's false positive buries a real email; a fraud model's false negative lets a criminal empty an account. Same matrix, wildly different stakes for the person on the other end.

Figuring out which error actually hurts starts with understanding the job the user hired your feature to do — the same lens our jobs-to-be-done complete guide uses to separate a feature's function from its emotional stakes. A spam filter's job is "keep my inbox usable"; a missed spam email is a minor tax on that job. A fraud model's job is "protect my money"; a missed fraud case is a complete failure of that job, not a minor one.

Mapping exactly where each error lands on the customer journey makes the cost difference concrete instead of abstract — a false positive that happens during onboarding (blocking a new user's first transaction) erodes trust differently than the same false positive six months into a relationship.

Four quick examples of the same asymmetry, in different domains:

1. Spam filter — a false positive hides a real email (occasionally costly, often just annoying); a false negative shows one spam message (mildly annoying). Favor precision.
2. Fraud detection — a false positive blocks a legitimate transaction (a support ticket, a frustrated customer); a false negative lets money walk out the door. Favor recall.
3. Medical screening test — a false positive means an unnecessary follow-up (cost and anxiety, but recoverable); a false negative means a missed disease. Favor recall, and accept a lot of follow-up tests.
4. Resume screening — a false positive wastes a recruiter's time on a bad-fit candidate; a false negative silently drops a great candidate who never even knows they were considered. Ambiguous, and worth debating explicitly rather than defaulting.

Precision and Recall, Defined Without the Greek Letters

Precision is TP divided by (TP + FP) — of everything you flagged as positive, what fraction was actually correct. Recall is TP divided by (TP + FN) — of everything that was actually positive, what fraction you caught. High precision means few false alarms; high recall means few misses.

A worked example makes this concrete. Say a fraud model reviews 1,000 transactions and flags 50 of them. Of those 50, 30 are truly fraudulent (TP) and 20 are legitimate (FP) — that's a precision of 30/50, or 60%. Somewhere in the 950 transactions it didn't flag, 10 more fraud cases slipped through (FN), so recall is 30 out of the true 40 fraud cases, or 75%.

Both numbers are simultaneously true, and both matter — just to different people in the room. A fraud analyst reviewing flagged cases cares about precision, because a low number means they're wasting their day on false alarms. A risk officer reporting to the board cares about recall, because a low number means real fraud is slipping through uncounted.

Notice, too, that precision and recall are computed over different denominators — precision divides by everything you flagged (50), recall divides by everything that was actually true (40). That's the whole reason they can move in opposite directions: a model can improve one without touching the other at all, simply by changing how many cases it's willing to flag in the first place.

Precision and recall almost always trade off against each other as you adjust how aggressively a model flags things — loosen the criteria and recall climbs while precision falls, tighten it and the reverse happens. Google's Machine Learning Crash Course walks through this exact relationship using the same formulas, and it's worth internalizing as a curve, not a single number: every operating point on that curve is a different bet about which mistake you'd rather make.

Why F1 Score Hides the Trade-off You Need to Own

F1 is the harmonic mean of precision and recall — a single number that punishes imbalance between the two, but says nothing about which one you actually need more of. Two models can post an identical F1 of 0.75 while one is a precise-but-blind fraud screen and the other is a trigger-happy alarm; F1 alone can't tell your stakeholders which one is safer to ship.

The formula is F1 = 2 × (precision × recall) / (precision + recall). Because it's a harmonic mean rather than a simple average, F1 drops sharply when either precision or recall is very low — a model with 95% precision and 10% recall scores an F1 of about 0.18, not the roughly 0.53 a plain average would suggest. That property makes F1 good at flagging severe imbalance, and bad at telling you which side of the imbalance you're on.

Statistician David Powers made a version of this argument in a widely cited 2011 paper on evaluation measures, contending that the F-measure was borrowed into classification from information retrieval without a principled reason for weighting precision and recall equally, and that it can even rank two classifiers in opposite order depending on which class you call "positive." That's a strong claim from someone who studies this for a living, and it holds up: F1 is a summary statistic, not a decision.

If you must compress precision and recall into one number, use F-beta and choose beta on purpose instead of defaulting to F1's implied 50/50 split:

• F2 weights recall twice as heavily as precision — a sane default for fraud, safety, or anything where a miss is the catastrophic outcome.
• F0.5 weights precision twice as heavily as recall — closer to what a spam filter or a low-stakes recommendation feature actually wants.
• F1 is the neutral midpoint, appropriate only when you've genuinely decided both errors cost the same — which is rarer than most dashboards assume.

Scikit-learn's evaluation tooling, including its precision_recall_curve function, is built around exactly this idea: precision and recall are reported as a curve across thresholds, not a single collapsed score, because the library's maintainers assume you'll pick the operating point rather than accept a default.

A Scenario: Deliberately Favoring Recall (and Tuning the Threshold)

Favor recall over precision whenever a missed positive is catastrophic and a false alarm is merely reviewable — a fraud model, an account-takeover detector, or a safety-content flag are the classic cases. You get there not by retraining the model but by lowering its decision threshold, trading a flood of false positives for almost no false negatives.

Here's how that plays out. Most classifiers don't output a flat yes/no — they output a probability, like 0.83, and a threshold converts that score into a decision. At a threshold of 0.5, an account-takeover detector might run at 92% precision and 40% recall: confident, but missing most of the real attacks. Lower the threshold to 0.15, and the same model might swing to 35% precision and 95% recall — noisy, but catching nearly everything.

Whether that swing is the right call depends entirely on what happens next to a flagged case. If a false positive means an account gets auto-blocked, a 35% precision rate is brutal — most flagged users did nothing wrong. If a false positive instead routes to a 30-second step-up verification (a text code, a security question) rather than an outright block, the cost of a false alarm drops close to zero, and pushing the threshold down to chase recall becomes an easy call.

Notice what didn't change anywhere in that example: the model itself. Threshold tuning is a product decision layered on top of a fixed model, which is exactly why it belongs to the PM and the risk or trust-and-safety lead, not just the data scientist. Plotting precision against recall across every possible threshold produces a precision-recall curve, and picking a point on that curve — not retraining, not adding features — is usually the fastest lever you have to shift the trade-off before launch.

The financial stakes behind that decision are real and well documented. The Association of Certified Fraud Examiners' biennial Report to the Nations has repeatedly found that organizations lose on the order of 5% of annual revenue to fraud — a large enough number that most fraud and trust-and-safety teams will happily accept dozens of false alarms to avoid a single missed theft.

One more wrinkle worth flagging: if the "classifier" flagging cases is actually an LLM prompt rather than a trained model — increasingly common for judgment calls like "is this support ticket urgent" or "does this listing look fraudulent" — the same threshold logic still applies to whatever confidence or rubric score the model returns. Read our guide on when to trust an LLM-as-judge before treating that score as ground truth, since an LLM's stated confidence doesn't calibrate the same way a trained classifier's probability output does.

Committing to a Threshold Before You Ship

A metric without a committed number is just an opinion. "Good recall" means nothing until someone writes down 90% or 60% — every classification-style feature needs an explicit precision target and an explicit recall target, agreed on before launch, tied to the actual cost of each error type for your specific product.

This is the same discipline our piece on numeric thresholds for failable evals argues for more broadly: an eval that can't fail on paper isn't an eval, it's theater. A spec that says "catch fraud reliably" hasn't committed to anything an engineer can test against; a spec that says "recall ≥ 90% at precision ≥ 40%, measured against a 200-example labeled set" has.

Prodinja's eval-plan step, part of the prototype's simulated evals-critique layer, is built around exactly this discomfort. When you describe a classification-style feature — a spam filter, a fraud flag, a churn-risk label — inside that flow, the template is designed to stop you until you've entered a numeric precision target and a numeric recall target, not just a vague pass/fail description. It won't run your model or compute your confusion matrix for you — that still depends on your own golden dataset and your own labeled outcomes — but it's built so you can't hand off a spec that says "catch fraud reliably" without also writing down what "reliably" means in TP, FP, and FN terms your engineering team can actually test.

Key Takeaways

• Precision and recall answer different questions. Precision asks "of what I flagged, how much was real?" Recall asks "of what was real, how much did I catch?"
• False positives and false negatives are never equally costly. A spam filter's false positive is a nuisance; a fraud model's false negative is a loss — cost depends on the product, not the math.
• F1 hides the trade-off, it doesn't resolve it. Two models can post an identical F1 with opposite strengths, so never greenlight a classification feature on F1 alone.
• Threshold tuning, not retraining, is usually how you shift the trade-off. Lowering the decision threshold trades precision for recall without touching the underlying model.
• F-beta lets you weight the trade-off on purpose. Use F2 when misses are the dangerous outcome, F0.5 when false alarms are, and reserve plain F1 for genuinely equal-cost errors.
• Every classification feature needs a committed numeric target — write down the precision and recall numbers you're aiming for before you ship, not after the first bad demo.
• None of this means anything without a labeled golden dataset. The confusion matrix is only as good as the ground truth you measured it against.