Forecasting Energy Demand When a Wrong Number Costs Megawatts

Forecasting energy demand well means shaping your model's error distribution around the asymmetric cost of being wrong, not just minimizing average error. A single under-forecast can force emergency purchases at scarcity prices, while over-forecasting wastes capacity at a smaller loss. Spec, evaluate, and choose your forecast type around that cost ratio.

Quick answer: Over- and under-forecasting are not mirror-image mistakes — they carry different dollar costs. Convert that cost ratio into a target quantile using pinball loss, reach for a probabilistic forecast when the ratio is far from 1:1, and backtest against real settlement rules, not just MAPE.

Why Optimizing for Accuracy Alone Misprices Risk

Accuracy metrics like MAPE and RMSE penalize every miss identically, but grid economics never do — an under-forecast that triggers a real-time scarcity purchase can cost many times more than an over-forecast that just strands committed capacity. Training a model on symmetric loss quietly bakes in the wrong incentive from day one.

Most wholesale power markets settle in two stages. A day-ahead forecast sets a commitment; a real-time market reconciles it against actual demand. Operators like PJM Interconnection and ERCOT price that gap using scarcity mechanisms — ERCOT's Operating Reserve Demand Curve can push real-time prices toward its system-wide cap (roughly $5,000/MWh) when reserves run thin.

Europe follows the same logic. Balancing markets coordinated through ENTSO-E rules and operators like the UK's National Grid ESO price imbalance to punish whichever direction threatens system reliability.

That asymmetry means the two forecast failure modes are not symmetric business problems:

• Under-forecasting — actual demand exceeds your number, so you scramble for last-minute supply, spinning reserve, or in extreme cases involuntary load shedding. Cost scales with scarcity pricing, which is designed by regulators to spike.
• Over-forecasting — you commit or generate more than needed, and the excess sells back at a depressed or negative price, or sits curtailed. Cost is real but bounded by the spread between contract and disposal price.

A PM who ships a forecasting feature and reports "MAPE improved from 4.1% to 3.6%" has told stakeholders nothing about which direction the remaining error leans, or what it costs. If you're building anywhere in the broader energy stack, it's worth grounding this in the wider landscape covered in our energy and climate tech complete guide, since forecasting sits inside a much bigger set of grid, renewables, and market-design constraints.

The Worked Example: Turning MAPE and Pinball Loss into Imbalance Dollars

Converting an accuracy metric into a dollar figure requires multiplying each hour's error by the price spread that direction of error actually incurs, then comparing that total against a model tuned for the cost ratio instead of average error. A model with slightly worse MAPE can carry a dramatically lower imbalance bill.

Take an illustrative (not a real settlement) 200 MW retail-load portfolio. The day-ahead contracted price is $45/MWh. When actual demand exceeds forecast, the entity buys the shortfall in real time at a scarcity-driven spread averaging $400/MWh above contract. When actual demand falls short, the entity sells the excess back at a spread of roughly $35/MWh below contract. Two candidate models, backtested over 100 hours:

Model B is "worse" on the metric most dashboards report, and better by more than half on the number Finance actually cares about. The mechanism behind that gap is pinball loss (also called quantile loss), which scores a forecast against a specific quantile τ rather than the mean:

L_τ(y, ŷ) = (y − ŷ) · τ,       if y ≥ ŷ  (under-forecast)
L_τ(y, ŷ) = (ŷ − y) · (1 − τ), if y < ŷ  (over-forecast)

The statistician Tilmann Gneiting's work on strictly proper scoring rules formalized why training against the right quantile — instead of the mean — produces forecasts that are honest about asymmetric stakes rather than merely "accurate." The target quantile itself comes from classic inventory theory: the newsvendor model's critical fractile, τ* = C_under / (C_under + C_over), where C_under and C_over are your per-unit costs of under- and over-forecasting.

Notice the fourth row: when over-forecasting is the expensive direction, the optimal target flips below the median. Most teams never ask this question — they train for the mean and assume symmetric cost was a modeling detail, not a product decision.

The Rubric: When a Probabilistic Forecast Beats a Point Forecast

Reach for a probabilistic forecast when the cost ratio between under- and over-forecasting is far from 1:1, when the forecast feeds an automated action with no human review, or when weather-driven volatility makes a single number misleading. Otherwise a well-calibrated point forecast, delivered on time, beats a fancier distribution nobody uses.

The decision isn't ideological — it's a function of five signals a PM can score in a single planning conversation:

A few implications worth spelling out:

1. Renewables push you toward probabilistic by default. Wind and solar variability is exactly the kind of volatility that makes a single point number fragile; ensemble and quantile-based methods are why national labs like NREL have spent years researching probabilistic solar and wind forecasting for grid operators managing that variability.
2. Regulatory submission and internal decision-making can use different formats. You can submit a point forecast for settlement while running the full distribution internally to size your risk buffer — the two aren't mutually exclusive.
3. "We'll add uncertainty bands later" is a roadmap trap. Retrofitting quantile output onto a model architected only for a point estimate is a rebuild, not a feature flag. Decide at the spec stage.

Why Offline Metrics Lie About Operational Cost

An aggregate accuracy score computed over a full backtest treats every hour as equally important, but the hours that matter operationally are the ones near a decision threshold — a thin reserve margin, a ramp-rate limit, a price-cap trigger — and those are usually a small, non-random slice of the dataset. A model can look great on MAPE and still be dangerously wrong exactly when it counts.

This is the same trap forecasting competitions have surfaced repeatedly. Spyros Makridakis's M-competitions (M4, M5) found that methods optimized for average error metrics like MASE or sMAPE often lose to simpler approaches once you isolate volatile or extreme periods. The tail is where the interesting product decisions live, and it's exactly what an aggregate score smooths over.

Textbook guidance from Rob Hyndman and George Athanasopoulos (Forecasting: Principles and Practice) makes the same point about quantile scoring: evaluate a forecast against the decision it informs, not against a generic error norm.

There's also a timing mismatch worth naming directly. Your forecast can update every five to fifteen minutes — a software clock — while the generation and reserve commitments it informs move on an hours-long, physically constrained cycle. That mismatch is a specific case of the pattern described in our piece on the hardware-software two-clock problem: the layer that can iterate fast is not the layer that has to live with the consequences.

Practical fix: build a cost-weighted backtest that replays historical forecasts through an approximation of your actual settlement rules, and report it segmented by the operating conditions that matter:

• Peak-load hours vs. off-peak hours
• Low-reserve-margin periods vs. normal conditions
• High-renewable-output hours vs. low-renewable-output hours
• Extreme-weather days vs. typical days

If your dashboard only shows one blended number, you are optimizing for the average day and getting graded on the worst one.

Spec'ing It: A PM Checklist for Asymmetric-Cost Forecasting

A forecasting feature spec should name the cost function before it names the model architecture, because the cost ratio determines the target quantile, the evaluation method, and the UX for human override. Skipping straight to "improve accuracy" hands the data science team an underspecified problem.

Before a forecasting feature moves past discovery, walk it through this checklist:

1. Define C_under and C_over explicitly, in dollars per unit, with Finance or the trading desk in the room — not as a data science assumption made in isolation.
2. Derive the target quantile from the critical fractile and revisit it whenever price spreads shift meaningfully (seasonally, or after a market design change).
3. Decide point vs. probabilistic output using the rubric above, matched to what the downstream system or team can actually consume.
4. Build a cost-weighted backtest that replays actual settlement logic, segmented by the threshold conditions where misses are expensive.
5. Alert on the cost-weighted metric, not the statistical one — a drift monitor watching RMSE will miss a model that's quietly drifting toward the expensive side of the distribution.
6. Design the override UX for the human in the loop: what does a dispatcher see when the forecast crosses a threshold band, and what's the one-click action?
7. Log the assumptions behind every forecast run — data source, weather model version, market condition, feature set — so that when a large miss happens, the root cause is traceable instead of buried in a retrained model nobody remembers the inputs to.

That last item is where most forecasting teams quietly lose the thread. Model versions change, weather feeds get swapped, a feature gets deprecated — and six months later nobody can reconstruct which assumption was live when a bad forecast cost real money.

Beyond the Grid: The Pattern Repeats, and Keeping Assumptions Visible

Asymmetric-cost forecasting is a transferable skill, not a grid-specific one, and the discipline that protects the dollar cost of a miss is the same discipline that keeps the reasoning behind a forecast from disappearing when a model gets retrained. Both problems reward the same habit: write the assumption down before you need it.

The Pattern Repeats Everywhere Capacity Is Constrained

Asymmetric forecast cost isn't a grid-specific quirk — it shows up anywhere a PM forecasts demand against a fixed or slow-moving supply of capacity, and the same spec discipline transfers directly. Recognizing the pattern early saves you from re-deriving it from scratch in a new vertical.

A few places the same math applies almost unchanged:

• Telecom network capacity — under-provisioning drops calls or spikes latency; over-provisioning sits idle as sunk capex. The telecom complete guide covers how capacity and reliability trade-offs shape telecom product decisions more broadly.
• Manufacturing and IIoT — a stockout halts a production line at a cost far beyond a unit of excess inventory sitting in a warehouse, which is why the manufacturing and IIoT complete guide treats demand and maintenance forecasting as a cost-asymmetry problem, not an accuracy contest.
• Automotive and mobility — EV charging and fleet-dispatch forecasts face the identical trade-off between stranded drivers and idle chargers, discussed further in the automotive and mobility complete guide.
• Media and streaming — under-provisioning CDN or compute capacity for a launch causes buffering and churn; over-provisioning burns cloud spend, a trade-off explored in the media and creator economy complete guide.

If you take one thing from the cross-industry comparison, take this: whenever you catch yourself defending a forecasting feature by quoting a single accuracy number, ask what it costs on each side of the miss. That question travels across every vertical listed above.

Keeping the Assumptions Behind a Forecast Visible

Forecasting is one of the clearest legitimate AI use cases in a PM's toolkit — but a forecast is only as trustworthy as the assumptions feeding it, and those assumptions tend to evaporate the moment a model gets retrained. This is a documentation problem as much as a modeling one.

When a miss happens, the question shouldn't be "what did the model do," it should be "which assumption was live, and was it still true." Keeping that trail visible, rather than buried inside a model artifact, turns a post-mortem from archaeology into a five-minute lookup.

Key Takeaways

• Under- and over-forecasting almost never cost the same; spec the cost ratio (C_under, C_over) before you spec the model.
• Use the newsvendor model's critical fractile (τ* = C_under / (C_under + C_over)) to convert that ratio into a target quantile for pinball loss.
• A model with worse MAPE can carry a dramatically lower imbalance bill if its errors are shaped toward the cheaper side of the cost curve.
• Choose probabilistic forecasts when the cost ratio is far from 1:1, the output feeds automation, or renewable-driven volatility makes a single number misleading; otherwise a timely point forecast is enough.
• Aggregate offline metrics like MAPE and RMSE hide the threshold-adjacent hours where misses are actually expensive — build a cost-weighted, threshold-segmented backtest instead.
• Log the assumptions behind each forecast run so a bad miss is traceable to its cause rather than lost when the model gets retrained.