Explainability When a Model Sets Someone's Premium: What PMs Owe the Customer and the Regulator

If a model raised a customer's premium, two people will ask why: the customer, who got an adverse-action notice, and the regulator, who reviewed your rate filing. Insurance pricing explainability means answering both — with a global account of which factors drive rates across the book, and a local account of why this specific policyholder got this specific number.

Quick Answer: Explainability for a rating model has two layers — global (which features drive rates overall, for regulatory filings) and local (why this customer's rate moved, for adverse-action reason codes). Ship both before the model goes live, not after a complaint.

Most PMs building rating engines still treat explainability as a documentation chore that happens after the model ships — a compliance team's problem, not a product requirement. That ordering is backwards, and it's expensive to reverse. A model that can't produce a reason code on demand is a model you can't legally use to raise someone's price in most U.S. states, and a model whose feature effects can't be summarized for a filing is a model your state insurance department will bounce back for more information — or reject outright.

Why Explainability Is a Regulatory Requirement, Not a UX Nicety

Explainability is legally required because insurance rating is one of the few consumer-facing uses of predictive models with an explicit, decades-old adverse-action disclosure regime attached to it. The Fair Credit Reporting Act (FCRA) requires specific reason codes whenever a credit-based factor contributes to an adverse action, and most state insurance codes extend equivalent notice requirements to any rating factor.

This isn't a novel AI-era obligation — it's an old rule meeting a new kind of model. When rating relied on a dozen tariff variables in a transparent linear formula, "why did my rate go up" had an easy answer: the underwriter could point to the factor and the coefficient. A gradient-boosted model with hundreds of engineered features and interaction terms doesn't have coefficients in that sense, which is exactly why the reason-code obligation gets harder to satisfy, not less applicable.

The Adverse-Action Chain, End to End

An adverse-action notice isn't a single letter — it's the last step in a chain that has to exist upstream of it. If any link is missing, the notice becomes fiction: text describing a process the model didn't actually go through.

1. Model produces a score or rate for an individual applicant or renewal.
2. A local attribution method identifies which inputs pushed that score up or down, and by roughly how much.
3. Attributions map to a fixed, pre-approved list of adverse-action reason codes (not free-text model internals).
4. The top 2-4 reasons get surfaced to the customer in plain language, per state notice requirements.
5. The same mapping is auditable — a regulator or a complainant's attorney can request the underlying attribution for a specific policy and get a consistent answer.

Skip step 2 and step 4 becomes guesswork. Skip step 5 and you have a notice you can't defend if challenged. This is why explainability has to be architected into the model pipeline, not retrofitted as a customer-service template — a point closely related to the fairness obligations covered in our guide to AI underwriting speed, fairness, and regulation.

Global Explainability: What Regulators Want to See in a Filing

Global explainability answers "what does this model do across the whole book," and it's what actuaries and regulators review during a rate filing, not what any individual customer sees. It typically takes the form of feature-importance rankings, partial-dependence summaries, and documented monotonicity constraints tying each factor to actuarially justified risk.

State rate filings — reviewed under NAIC (National Association of Insurance Commissioners) model laws that most states adopt in some form — require insurers to demonstrate that every rating variable is actuarially justified and doesn't produce unfairly discriminatory outcomes. A regulator reading a filing isn't asking about any one policyholder; they're asking whether the model's overall behavior, aggregated across thousands of profiles, is defensible.

What a Global Explainability Package Typically Includes

A model that can't produce this package isn't unexplainable in some abstract sense — it's unfilable. Some state departments of insurance have signaled reluctance to approve rating plans built on opaque ensemble models (gradient-boosted trees, deep nets) without a documented, model-agnostic explainability layer sitting on top, precisely because the raw model provides none of the rows in the table above on its own.

Local Explainability: Reason Codes for a Single Customer

Local explainability answers "why did this customer get this rate," and it's produced with attribution methods that decompose one prediction into per-feature contributions, then map those contributions to a small, pre-approved set of adverse-action reason codes. This is the layer that actually reaches the customer.

SHAP (SHapley Additive exPlanations), built on Lloyd Shapley's cooperative game theory and popularized for ML by Scott Lundberg and Su-In Lee, is the dominant technique here: it distributes a prediction's deviation from a baseline across each input feature, additively and consistently, so the contributions sum to the actual prediction. LIME (local interpretable model-agnostic explanations) is a common alternative, fitting a simple local surrogate model around one prediction.

Neither tool speaks in a customer's or a regulator's language on its own. The product work is the translation layer: mapping "the SHAP value for credit_tier_bucket was +$38/month" into an approved reason code like "Prior insurance history" or "Credit-based insurance score," worded per each state's required disclosure taxonomy.

Global vs. Local Explainability, Side by Side

Both layers have to exist before launch — a filing without global evidence won't get approved, and a live model without a local attribution path can't legally generate adverse-action notices the moment it goes live.

Why a Black-Box GBM Can Become Unfilable

A gradient-boosted model can become effectively unfilable when its complexity — hundreds of trees, deep interaction effects, no closed-form coefficients — makes it impossible to produce the global monotonicity and feature-attribution documentation a state regulator requires, even though the model's raw predictive accuracy is fine.

This is a real, structural tension, not a hypothetical one. GBMs and similar ensembles routinely outperform generalized linear models (GLMs) on lift and Gini metrics in back-tests — the actuarial literature on machine learning in pricing (much of it published through the Casualty Actuarial Society) documents this repeatedly. But accuracy on holdout data and defensibility in a filing are different tests, and a model can pass the first while failing the second.

Three Ways Teams Resolve the Tension

1. Constrained or monotonic GBMs — train the ensemble with explicit monotonicity constraints per feature, so the filing package can truthfully claim the model behaves the way an actuary expects, at some cost to raw accuracy.
2. Two-model architectures — use the black-box model as a research/validation benchmark, but file and deploy a GLM (or GAM) whose coefficients are natively interpretable, treating the GBM's lift as a signal for where to add interaction terms manually.
3. Full attribution + surrogate documentation — file the GBM as-is, but pair it with a SHAP-based global summary, per-feature partial dependence plots, and a surrogate GLM fit to the GBM's outputs as an interpretability check, submitted alongside the filing.

None of these is free. Option 1 sacrifices some accuracy for filability. Option 2 sacrifices some accuracy for simplicity and speed to filing. Option 3 preserves the GBM but adds real engineering and documentation overhead that has to be planned for, not discovered during a filing rejection.

Trust as a Second, Separate Reason to Build This

Beyond the regulatory floor, explainability is also a trust lever: customers who receive a specific, legible reason for a rate increase are measurably more likely to accept it than customers who get a vague or generic notice, based on research on procedural fairness in decision communication.

Procedural-justice research — going back to Tom Tyler's work on why people accept unfavorable decisions — consistently finds that perceived fairness of the process matters as much as the outcome itself. A customer who understands "your rate increased because your vehicle's garaging ZIP code moved into a higher-loss territory" is in a fundamentally different position than one who gets "your rate has changed due to underwriting factors." The first is a fact they can evaluate or contest; the second reads as a shrug.

This matters commercially, not just legally. A rate increase paired with a specific, verifiable reason reduces the odds a customer escalates to a regulator complaint or a public review — both of which cost more in remediation time than the explainability infrastructure would have cost to build up front. The connection between decision transparency and how customers experience a moment of friction runs through the same territory as claims automation and the human moment and the broader arc mapped in the customer journey — a rate increase is as much an emotional inflection point as a claim denial.

Making Explainability a Hard Hand-Off Requirement

Treat explainability as a product requirement enforced at the same gate as data privacy or accessibility — a pricing feature doesn't clear review without a documented answer to "what's the reason-code path for this factor." That means writing it into the spec before a single line of model code ships, not appending it after a regulator asks.

Practically, a rating-model spec worth reviewing should require, at minimum:

• A named local attribution method (SHAP, LIME, or equivalent) with a documented mapping to the state's approved adverse-action reason-code list.
• A global explainability package — feature importance, partial dependence, monotonicity constraints — versioned alongside the model, not produced ad hoc at filing time.
• A model-change trigger that re-runs both packages whenever the model is retrained, so explainability artifacts never silently drift out of sync with the deployed model.
• An owner — pricing, compliance, or product — accountable for the reason-code mapping staying current as state disclosure requirements change.

Building this into intake is cheaper than every alternative. Compare it against the broader landscape of algorithmic accountability across insurtech functions in our complete guide to insurtech, and against the parallel transparency tension in fraud detection false-positive tradeoffs, where an unexplainable flag creates the same customer-trust and regulatory exposure as an unexplainable rate.

Key Takeaways

• Explainability is a legal floor, not a feature choice — FCRA and state insurance codes require adverse-action reason codes whenever a rating factor moves a price.
• Global explainability serves the filing: feature importance, partial dependence, and monotonicity documentation are what regulators actually review.
• Local explainability serves the customer: SHAP- or LIME-style attributions, translated into a fixed, pre-approved reason-code taxonomy.
• A black-box GBM can be accurate and still unfilable if it can't produce the global documentation a state regulator requires — plan for constrained models, a GLM pairing, or full attribution documentation from day one.
• Trust and compliance point the same direction: specific, legible reasons reduce complaint escalation as much as they satisfy the notice requirement.
• Build the reason-code path into the spec before the model ships — a readiness gate at hand-off is cheaper than a filing rejection or a regulator inquiry after launch.