Forecast evaluation and model selection
Crédits ECTS :
3
Heures de cours :
18
Heures de TD :
0
Langue :
Français
Objectif
The course (18 hours) provides tools and techniques to evaluate the accuracy model based forecasts and operate model selection. The tools and techniques can be applied to a variety of settings ranging from macroeconomics to finance, although particular focus is devoted to financial applications. The course has several specific objectives :
1- Discuss mathematical/statistical tools for forecast evaluation and model selection for scalar, vector and matrix valued outcomes and lay their theoretical foundation
2- Provide clear definition and interpretation of the metrics used to assess forecast accuracy
3- Focus on problems arising in forecast evaluation of latent target variables with emphasis on the case of volatility forecasts evaluation
Slides and notes are distributed in class. References are provided below. The exam is written, 2 hours, closed book.
Plan
Measures of forecast accuracy : Notion of empirical ranking. Definition of statistical loss functions (in scalar/vector/matrix spaces), construction and interpretation. Definition of economic loss functions (Minimum variance portfolio, Option pricing accuracy). Regression-based evaluation of predictive accuracy. Observable vs. Latent target variables. Evaluation of nested models : Estimation sampling schemes (Expanding vs. rolling window). Structural breaks/time varying parameters. Absolute vs. conditional forecasting performance. Tests of Forecast Comparison – Theory and Hypotheses : Single hypothesis (Diebold-Mariano, West, Clark-McCracken, Giacomini-White, Mincer-Zarnowitz), Multiple hypotheses (Superior Predictive Ability, Model Confidence Set). Simple vs. Composite hypotheses (testing equalities vs. weak inequalities). Evaluation of latent target variables forecasts (Volatility forecast evaluation) : Introduction to non.parametric ex-post measures of volatility (realized variance), microstructure noise, observation frequency, jump detection and synchronicity. Consistency of forecasts ranking under statistical and economic loss functions. Sufficient and necessary conditions for ranking consistency. Additional topic (depending on time): Predictive density evaluation (Diebold-Gunther-Tay, Corradi-Swanson, Amisano-Giacomini).
Références
Andersen T, Bollerslev T, Diebold F, Labys P. (2003) Modeling and forecasting realized volatility. Econometrica 71, 579-625
Barndorff-Nielsen O, Hansen P, Lunde A, Shephard N. (2008) Designing realized kernels to measure the ex post variation of equity prices in the presence of noise. Econometrica, 76 :1481-1536
Clark, T. and M. McCracken (2015), Nested Forecast Model Comparisons : A New Approach to Testing
Equal Accuracy, J. of Econometrics, Vol. 186, 160-177
Clark, T. and M. McCracken (2005), The Power of Tests of Predictive Ability in the Presence of Structural Breaks, J. of Econometrics 124(1), 1-31
Diebold. F, Mariano R. (1995), Comparing Predictive Accuracy, J. of Business and Economic Statistics 13
Elliott G., Timmermann A, (2008). Economic forecasting. J. of Economic Literature 46, 3-56.
Giacomini, R. and H. White (2006), Tests of Conditional Predictive Ability, Econometrica 74(6).
Hansen P. (2006) A test for superior predictive ability, J. Business and Economic Statistics, 23 :365-380.
Hansen P, Lunde A, Nason J. (2011) The model confidence set, Econometrica, 79 :453-497.
Hansen P, Lunde A. (2006), Consistent ranking of volatility models. J. of Econometrics, 131 :97-121
Laurent S, Rombouts JVK, Violante F. (2013), On loss functions and ranking forecasting performances of multivariate volatility models, J. of Econometrics 173 1-10
Mincer, J. and V. Zarnowitz (1969), The Evaluation of Economic Forecasts, in J. Mincer, ed., Economic
Forecasts and Expectations, 3-46
Patton A. (2011), Volatility forecast comparison using imperfect volatility proxies. J. of Econometrics, 160
West K, McCracken M. (1998), Regression-Based Tests of Predictive Ability, Int. Economic Review, 39
White H. (2000) Reality check for data snooping. Econometrica, 68 :1097-1126
Corradi, V. and N. Swanson (2001) Predictive Density Evaluation, in : Granger, C., G. Elliott and A.
Timmermann (eds.), Handbook of Economic Forecasting Vol. 1
Amisano, G. and R. Giacomini (2007), Comparing Density Forecasts via Weighted Likelihood Ratio Tests, J. of Business and Economic Statistics 25, 177-190