The evaluation of extrapolative forecasting methods

Examining how a forecasting method should be chosen based on analyzing alternative loss functions concludes that exponential smoothing and ‘naive’ models, previously thought to be ‘robust’ performers, forecast poorly for the particular set of time series under analysis, whatever error measure is used.

Abstract

Extrapolative forecasting methods are widely used in production and inventory decisions. Typically many hundreds of series are forecast and the cost-effectiveness of the decisions depends on the accuracy of the forecasting method(s) used. This paper examines how a forecasting method should be chosen based on analyzing alternative loss functions. It is argued that a population of time series must be evaluated by time period and by series. Of the alternative loss functions considered, only the geometric root mean squared error is well-behaved and has a straightforward interpretation. The paper concludes that exponential smoothing and ‘naive’ models, previously thought to be ‘robust’ performers, forecast poorly for the particular set of time series under analysis, whatever error measure is used. As a consequence, forecasters should carry out a detailed evaluation of the data series, as described in the paper, rather than relying on a priori analysis developed from earlier forecasting competitions.