Robust inference under timevarying volatility: A real-time evaluation of professional forecasters

Loading...
Thumbnail Image

Date

2021

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

In many forecast evaluation applications, standard tests (e.g., Diebold and Mariano, 1995) as well as tests allowing for time-variation in relative forecast ability (e.g., Giacomini and Rossi, 2010) build on heteroskedasticity-and-autocorrelation consistent (HAC) covariance estimators. Yet, the finite-sample performance of these asymptotics is often poor. "Fixed-b" asymptotics (Kiefer and Vogelsang, 2005), used to account for long-run variance estimation, improve finitesample performance under homoskedasticity, but lose asymptotic pivotality under time-varying volatility. Moreover, loss of pivotality due to time-varying volatility is found in the standard HAC framework in certain cases as well. We prove a wild bootstrap implementation to restore asymptotically pivotal inference for the above and new CUSUM- and Cramér-von Mises based tests in a fairly general setup, allowing for estimation uncertainty from either a rolling window or a recursive approach when fixed-b asymptotics are adopted to achieve good finite-sample performance. We then investigate the (time-varying) performance of professional forecasters relative to naive no-change and model-based predictions in real-time. We exploit the Survey of Professional Forecasters (SPF) database and analyze nowcasts and forecasts at different horizons for output and inflation. We find that not accounting for time-varying volatility seriously affects outcomes of tests for equal forecast ability: wild bootstrap inference typically yields convincing evidence for advantages of the SPF, while tests using non-robust critical values provide remarkably less. Moreover, we find significant evidence for time-variation of relative forecast ability, the advantages of the SPF weakening considerably after the "Great Moderation".

Description

Table of contents

Keywords

forecast evaluation, bootstrap, structural breaks, HAC estimation, hypothesis testing

Citation