Robust inference under timevarying volatility: A real-time evaluation of professional forecasters

Abstract

In many forecast evaluation applications, standard tests (e.g., Diebold and Mariano, 1995) as well as tests allowing for time-variation in relative forecast ability (e.g., Giacomini and Rossi, 2010) build on heteroskedasticity-and-autocorrelation consistent (HAC) covariance estimators. Yet, the finite-sample performance of these asymptotics is often poor. "Fixed-b" asymptotics (Kiefer and Vogelsang, 2005), used to account for long-run variance estimation, improve finitesample performance under homoskedasticity, but lose asymptotic pivotality under time-varying volatility. Moreover, loss of pivotality due to time-varying volatility is found in the standard HAC framework in certain cases as well. We prove a wild bootstrap implementation to restore asymptotically pivotal inference for the above and new CUSUM- and Cramér-von Mises based tests in a fairly general setup, allowing for estimation uncertainty from either a rolling window or a recursive approach when fixed-b asymptotics are adopted to achieve good finite-sample performance. We then investigate the (time-varying) performance of professional forecasters relative to naive no-change and model-based predictions in real-time. We exploit the Survey of Professional Forecasters (SPF) database and analyze nowcasts and forecasts at different horizons for output and inflation. We find that not accounting for time-varying volatility seriously affects outcomes of tests for equal forecast ability: wild bootstrap inference typically yields convincing evidence for advantages of the SPF, while tests using non-robust critical values provide remarkably less. Moreover, we find significant evidence for time-variation of relative forecast ability, the advantages of the SPF weakening considerably after the "Great Moderation".