In search of the ideal measure of accuracy for subnational demographic forecasts

The MAPE-R developed by the authors is lower than the MAPE based on the transformed errors and is more consistent with a robust estimator of location.

Abstract

We examine nonlinear transformations of the forecasterror distribution in hopes of finding a summary errormeasure that is not prone to an upward bias and usesmost of the information about that error. MAPE, thecurrent standard for measuring error, often overstatesthe error represented by most of the values becausethe distribution underlying the MAPE is right skewedand truncated at zero. Using a modification to theBox-Cox family of nonlinear transformations, wetransform these skewed forecast error distributionsinto symmetrical distributions for a wide range ofsize and growth rate conditions. We verify thissymmetry using graphical devices and statisticaltests; examine the transformed errors to determine ifre-expression to the scale of the untransformed errorsis necessary; and develop and implement a procedurefor the re-expression. The MAPE-R developed by ourprocess is lower than the MAPE based on theuntransformed errors and is more consistent with arobust estimator of location.