Which of the following is used to describe the degree of error in a forecasting model?

This appendix provides background and formulas for the error metrics used in the Forecast Scorecard dashboard. The errors are always calculated at the lowest level - typically item/store/week, and then averaged at the intersection of the dashboard tiles. For GA this intersection is subclass/district, but it can be configured at implementation time. The evaluation of the forecast is done over a window starting today and looking back a configurable number of periods. All errors are calculated for the system-generated forecast as well as the user adjusted forecast.

The following are the error metrics:

• Mean Absolute Percent Error

• Root Mean Squared Error

• Mean Absolute Error

• Forecast Bias

• Percent Adjusted

Mean Absolute Percent Error

The percentage error of a forecast observation is the difference between the actual POS value and the forecast value, divided by the actual POS value. The result of this calculation expresses the forecast error as a percentage of the actual value. The Mean Absolute Percentage Error statistic measures forecast accuracy by taking the average of the sum of the absolute values of the percentage error calculations across all observations. This method is useful when comparing the accuracy of forecasts for different volume products (it normalizes error by volume).

Root Mean Squared Error

This is the square root of the Mean Squared Error. The Root Mean Squared Error is one of the most commonly used measures of forecast accuracy because of its similarity to the basic statistical concept of a standard deviation. It evaluates the magnitude of errors in a forecast on a period-by-period basis, and it is best used to compare alternative forecasting models for a given series.

Mean Absolute Error

The absolute error of a forecast observation is the absolute value of the difference between the forecast value and the actual POS value. The Mean Absolute Error statistic is a measure of the average absolute error. This is calculated by summing the absolute errors for all observations and then dividing by the number of observations to obtain the average. Mean Absolute Error gives you a better indication of how the forecast performed period by period because the absolute value function ensures that negative errors in one period are not canceled out by positive errors in another. Mean Absolute Error is most useful for comparing two forecast methods for the same series.

Forecast Bias

Forecast BIAS is described as a tendency to either:

• over-forecast (meaning, more often than not, the forecast is more than the actual)

• under-forecast (meaning, more often than not, the forecast is less than the actual).

A desired property of a forecast is that it is not biased.

Percent Adjusted

This number represents the count of adjusted forecast values divided by the total count of forecast values. A high percentage indicates that the users heavily adjust the forecasts.

Measurement of Error

Several of the common terms used to describe the degree of error associated with forecasting are standard error, mean squared error (or variance), and mean absolute deviation. In addition. tracking signals may be used to indicate the existence of any positive or negative bias in the forecast. Standard error is discussed in the section on linear regression later in the chapter. Since the standard error is the square root of a function, it is often more convenient to use the function itself. This is called the mean square error, or variance.

The mean absolute deviation (MAD) was at one time very popular but subsequently was ignored in favor of the standard deviation and standard error measures. In recent years however MAD has made a comeback because of its simplicity and usefulness in obtaining tracking signals. MAD is the average error in the forecasts. u ing absolute values. It is valuable because MAD, like the tan dard deviation, measures the dispersion (or variation) of observed values around some expected value. MAD is computed using the differences between the actual demand and the forecast demand without regard to whether it is negative or positive. It therefore is equal to the sum of the absolute deviations divided by the number of data points or stated in equation form:

When the errors that occur in the forecast are normally distributed (which is usually assumed to be the case), the mean absolute deviation relate to standard division as

Conversely, I MAD 0.8 standard deviation The standard deviation is the larger measure. If the MAD for a set of points was found to be 60 units. then the standard deviation would be 75 units. And. in the usual statistical manner, if control limits were set at ±3 standard deviations (or ±3.75 MADs). then 99.7 percent of the points would fall within these limits. (See Exhibit 9.9.)

A tracking signal is a measurement that indicates whether the forecast average is keeping pace with any genuine upward or downward changes in demand. As used in forecasting the tracking signal is the number of mean absolute deviations that the forecast value is above or below the actual occurrence. Exhibit 9.9 shows a normal distribution with a mean of zero and a MAD equal to one. Thus if we compute a tracking signal and find it equal to 2 we can conclude that the forecast model is providing forecasts that are quite a bit above the mean of the actual occurrences. A tracking signal can be calculated using the arithmetic sum of forecast deviations divided by the mean absolute deviation, or

It is important to note that while the MAD, being an absolute value, is always positive, the tracking’signal can take on positive and negative values. Exhibit 9.10 illustrates the procedure for computing MAD and the tracking signal for a six-month period where the forecast had been set at a constant 1.000 and the actual demands that occurred are as shown. In this example, the forecast, Rutherford average, was off by 66.7 units and the tracking signal was equal to 3.3 mean absolute deviations. . We can obtain a better interpretation of the MAD and tracking signal by plotting the points on a graph. While not completely legitimate from a sample size standpoint. we plotted each month in Exhibit 9.11 to show the drifting of the tracking signal. Note that it drifted from -1MAD to +3.3 MADs. This occurred because the actual demand was greater than the forecast in four of the six periods. If the actual demand doesn’t fall below the forecast to offset the continual positive RSFE, the tracking signal would continue to rise and we ‘ould conclude that the assumption that demand ~ 1,000 is a bad forecast. When the tracking signal exceeds a pre established limit (for example, ±2.0 or ±3.0), the manager should n eider changing the forecast model or the value of ex. Acceptable limits for the tracking signal depend on the size of the demand being forecast (high-volume or high-revenue items should be monitored frequently) and the amount

Which of the following is used to describe the degree of error of a forecast?

Which measurement of error can be used to detect forecast bias?

Which one of the given options is a measure of overall forecasting errors?

What two techniques can be used to monitor forecast errors?