Forecast accuracy by week: window, bias and forecast value added
From weekly forecast and actual for a SKU or a group calculates the WAPE error and the bias overall and in a rolling window, the tracking signal, the comparison with a naive forecast (last week’s actual) and weeks with an error above your tolerance.
At a glance
- What question does it answer?
- How accurate the forecast is by week, whether it is biased and whether it beats a simple repeat.
- What data is needed?
- Forecast and actual by week (better the actual demand).
- How to read the result?
- On the default data (10 weeks) WAPE is 11.0%, bias minus 0.5%, the tracking signal minus 0.4. In the 4-week window WAPE is 10.3%, bias minus 3.4%. The naive forecast (last week’s actual) gives WAPE 20.4%, forecast value added 9.4 pp: the forecast is clearly better than a simple repeat. Example on the default values: WAPE over all weeks: 11%; Bias over all weeks: −0.5%; WAPE in the window: 10.3% · last 4 weeks; Bias in the window: −3.4%. The figures are illustrative: enter your own to see your result.
Why and how to reason
A forecast is useful only if it is better than a simple rule, and you need to know exactly how it errs. WAPE is the sum of absolute errors divided by the sum of actuals: 115 out of 1,045 units, that is 11%. It does not let large and small items err “in different directions” and hide the error. Bias shows direction: a sum of forecasts of 1,040 against actuals of 1,045 gives minus 0.5%, a slight under-forecast. The tracking signal is the accumulated error in units of mean error: a large value means a systematic skew. Finally, forecast value added (FVA) compares with the naive rule “same as last week”: its WAPE is 20.4%, and the forecast is better by 9.4 points. If a forecast is no better than the naive one, it adds no value.
When to use it
Use it weekly for key SKUs or groups and after every change of forecast method. Do not use it for series with many zeros: intermittent demand has its own calculation. Assumptions: forecast and actual are compared for the same periods and in the same units, you choose the window, the weekly error tolerance is yours. It suits a weekly planning meeting well: one page shows where the forecast errs and in which direction.
How to read the result
Look at pairs: WAPE over all time (11%) and in the window (10.3%) show whether accuracy is worsening or improving. Bias in the window of minus 3.4% says the last weeks were under-forecast: raise the level. FVA of +9.4 points confirms the value of the method. The largest error (20 units) and the weeks above tolerance show where to investigate causes (promotion, stock-out, data failure) instead of changing the model. If the tracking signal is far from zero, look for a systematic cause. Set a rule: if FVA is negative two weeks in a row, return to the naive forecast and look for the cause until the model becomes better.
Common mistakes
Typical trap: computing error against the forecast instead of the actual, comparing WAPE across aggregation levels (SKU and group), using sales instead of demand in a stock-out period, ignoring bias while enjoying a small error. A small error with a bias still leads to excess or missing stock. Do not blame the model for the error of a promotion week it did not know about: mark such weeks first, otherwise WAPE punishes the wrong thing.
What data is needed?
- Forecast and actual by week (better the actual demand).
- The window in weeks.
- The weekly error tolerance (optional).
Inputs
| Field | Unit | Value in the example | Meaning |
|---|---|---|---|
| Rolling window, weeks | weeks | 4 | How many latest weeks the rolling measure takes. If the table has fewer weeks, all are taken. The example value is illustrative. |
| Tolerable weekly error | % | — | The weekly error as a percentage of its actual, above which the week counts as a miss. Your own value: There is no single standard in Russia: enter your own figure. Optional. Empty: the field is ignored (counted as zero in sums). |
How it is calculated
| Step | Formula |
|---|---|
| WAPE | sum |forecast − actual| ÷ sum of actuals |
| Bias | (sum of forecasts − sum of actuals) ÷ sum of actuals |
| Tracking signal | sum of errors ÷ mean |error| |
| FVA | naive WAPE − forecast WAPE, weeks from the second |
Worked numbers
Default values from the app; in the interactive version you enter your own.
- WAPE over all weeks
- 11 %Sum of absolute errors ÷ sum of actuals.
- Bias over all weeks
- −0.5 %Plus: forecast too high; minus: too low.
- WAPE in the window
- 10.3 % · last 4 weeks
- Bias in the window
- −3.4 %
- Tracking signal
- −0.4 mean errorsCumulative error ÷ mean absolute deviation; far from zero means a systematic bias. The limit is yours: there is no single standard.
- WAPE of the naive forecast
- 20.4 %Week forecast = last week’s actual; from week two.
- Forecast value added (FVA), p.p.
- 9.4Naive WAPE minus forecast WAPE on the same weeks: below zero means the “forecast” is worse than a simple repeat.
- Biggest error
- 20 · Week 3
- Weeks above tolerance
- no tolerance set
Data as a table
| Measure | WAPE |
|---|---|
| WAPE, all weeks | 11 |
| WAPE in the window | 10.3 |
| Naive WAPE | 20.4 |
Rolling window by week
| Window ends | WAPE | Bias |
|---|---|---|
| Week 4 | 11.6 % | 2.3 % |
| Week 5 | 11.4 % | −2.3 % |
| Week 6 | 12.2 % | 2.4 % |
| Week 7 | 9.3 % | −4.7 % |
| Week 8 | 11.4 % | 1.3 % |
| Week 9 | 10 % | 2.5 % |
| Week 10 | 10.3 % | −3.4 % |
Here is a weekly window and a comparison with a naive forecast; monthly bias, the tracking signal by month and the demand and supply balance: the “S&OP: forecast bias, error and demand and supply balance” calculation. There is no single accuracy standard in Russia: the thresholds are yours.
Nuances and limits
- Take the forecast made before the week, not a recalculated one.
- Sales understate demand when goods run out.
- A short window: on small sales the error is noisy.
- Monthly S&OP and the demand and supply balance: the “S&OP: forecast bias…” calculation.