Forecast Accuracy Measures: Results from Simple Moving Average Methods on Independent Normal Time Series

Abstract

Comparing the forecasting accuracy of techniques using data sets from different disciplines is often very difficult and may result in disagreements on which techniques produce more accurate forecasts. Hence, a model-based simulation study is useful for gaining insight into the quality of these forecast accuracy measures. This study compares eleven forecast accuracy measures of forecasting errors generated from Simple Moving Average methods on independent normal time series with moderate coefficients of variation (c.v.). Contrary to the findings of other authors, the traditional measures MSE, RMSE, and GMAE performed as well as the MAD when Simple Moving Average methods were applied to independent normal time series. Moreover, the highly recommended GMRAE, MAPE, unbiased MUAPE and MdUAPE proposed in recent publications cannot and did not perform well in distinguishing different Simple Moving Average methods applied to independent normal time series. This study also concurs with the recommendations in recent publications that the Mean Absolute Scaled Error (MASE) should become the standard measure for comparing forecast accuracy. If percentage error is a concern for comparisons among time series with different units, the MdAPE and/or MASE, not the MAPE, is recommended.