Syntha Lab RU

Forecast accuracy by week: window, bias and forecast value added

From weekly forecast and actual for a SKU or a group calculates the WAPE error and the bias overall and in a rolling window, the tracking signal, the comparison with a naive forecast (last week’s actual) and weeks with an error above your tolerance.

KPI dictionary and common definitions For whom: Planners, demand analysts and category managers when reviewing a forecast.

Calculate on your data →

At a glance

What question does it answer?
How accurate the forecast is by week, whether it is biased and whether it beats a simple repeat.
What data is needed?
Forecast and actual by week (better the actual demand).
How to read the result?
On the default data (10 weeks) WAPE is 11.0%, bias minus 0.5%, the tracking signal minus 0.4. In the 4-week window WAPE is 10.3%, bias minus 3.4%. The naive forecast (last week’s actual) gives WAPE 20.4%, forecast value added 9.4 pp: the forecast is clearly better than a simple repeat. Example on the default values: WAPE over all weeks: 11%; Bias over all weeks: −0.5%; WAPE in the window: 10.3% · last 4 weeks; Bias in the window: −3.4%. The figures are illustrative: enter your own to see your result.

Why and how to reason

A forecast is useful only if it is better than a simple rule, and you need to know exactly how it errs. WAPE is the sum of absolute errors divided by the sum of actuals: 115 out of 1,045 units, that is 11%. It does not let large and small items err “in different directions” and hide the error. Bias shows direction: a sum of forecasts of 1,040 against actuals of 1,045 gives minus 0.5%, a slight under-forecast. The tracking signal is the accumulated error in units of mean error: a large value means a systematic skew. Finally, forecast value added (FVA) compares with the naive rule “same as last week”: its WAPE is 20.4%, and the forecast is better by 9.4 points. If a forecast is no better than the naive one, it adds no value.

When to use it

Use it weekly for key SKUs or groups and after every change of forecast method. Do not use it for series with many zeros: intermittent demand has its own calculation. Assumptions: forecast and actual are compared for the same periods and in the same units, you choose the window, the weekly error tolerance is yours. It suits a weekly planning meeting well: one page shows where the forecast errs and in which direction.

How to read the result

Look at pairs: WAPE over all time (11%) and in the window (10.3%) show whether accuracy is worsening or improving. Bias in the window of minus 3.4% says the last weeks were under-forecast: raise the level. FVA of +9.4 points confirms the value of the method. The largest error (20 units) and the weeks above tolerance show where to investigate causes (promotion, stock-out, data failure) instead of changing the model. If the tracking signal is far from zero, look for a systematic cause. Set a rule: if FVA is negative two weeks in a row, return to the naive forecast and look for the cause until the model becomes better.

Common mistakes

Typical trap: computing error against the forecast instead of the actual, comparing WAPE across aggregation levels (SKU and group), using sales instead of demand in a stock-out period, ignoring bias while enjoying a small error. A small error with a bias still leads to excess or missing stock. Do not blame the model for the error of a promotion week it did not know about: mark such weeks first, otherwise WAPE punishes the wrong thing.

What data is needed?

  • Forecast and actual by week (better the actual demand).
  • The window in weeks.
  • The weekly error tolerance (optional).

Inputs

FieldUnitValue in the exampleMeaning
Rolling window, weeksweeks4How many latest weeks the rolling measure takes. If the table has fewer weeks, all are taken. The example value is illustrative.
Tolerable weekly error%—The weekly error as a percentage of its actual, above which the week counts as a miss. Your own value: There is no single standard in Russia: enter your own figure. Optional. Empty: the field is ignored (counted as zero in sums).

How it is calculated

StepFormula
WAPEsum |forecast − actual| ÷ sum of actuals
Bias(sum of forecasts − sum of actuals) ÷ sum of actuals
Tracking signalsum of errors ÷ mean |error|
FVAnaive WAPE − forecast WAPE, weeks from the second

Worked numbers

Default values from the app; in the interactive version you enter your own.

WAPE over all weeks
11 %Sum of absolute errors ÷ sum of actuals.
Bias over all weeks
−0.5 %Plus: forecast too high; minus: too low.
WAPE in the window
10.3 % · last 4 weeks
Bias in the window
−3.4 %
Tracking signal
−0.4 mean errorsCumulative error ÷ mean absolute deviation; far from zero means a systematic bias. The limit is yours: there is no single standard.
WAPE of the naive forecast
20.4 %Week forecast = last week’s actual; from week two.
Forecast value added (FVA), p.p.
9.4Naive WAPE minus forecast WAPE on the same weeks: below zero means the “forecast” is worse than a simple repeat.
Biggest error
20 · Week 3
Weeks above tolerance
no tolerance set
Forecast error: all weeks, the window and the naive forecast
Bar chart: WAPE, all weeks 11, WAPE in the window 10.3, Naive WAPE 20.40510152025WAPE, all weeksWAPE, all weeksWAPE: 1111WAPE in the windowWAPE in the windowWAPE: 10.310.3Naive WAPENaive WAPEWAPE: 20.420.4%
Data as a table
MeasureWAPE
WAPE, all weeks11
WAPE in the window10.3
Naive WAPE20.4

Rolling window by week

Window endsWAPEBias
Week 411.6 %2.3 %
Week 511.4 %−2.3 %
Week 612.2 %2.4 %
Week 79.3 %−4.7 %
Week 811.4 %1.3 %
Week 910 %2.5 %
Week 1010.3 %−3.4 %

Here is a weekly window and a comparison with a naive forecast; monthly bias, the tracking signal by month and the demand and supply balance: the “S&OP: forecast bias, error and demand and supply balance” calculation. There is no single accuracy standard in Russia: the thresholds are yours.

Nuances and limits

  • Take the forecast made before the week, not a recalculated one.
  • Sales understate demand when goods run out.
  • A short window: on small sales the error is noisy.
  • Monthly S&OP and the demand and supply balance: the “S&OP: forecast bias…” calculation.

Terms used

Calculate on your data →