Forecast evaluation and model selection
ECTS:
3
Course Hours:
18
Tutorials Hours:
0
Language:
French
Objective
The course is designed to acquire tools and techniques to evaluate the accuracy model based forecasts and operate model selection in a variety of settings. The tools and techniques can be applied to a variety of settings ranging from macroeconomic forecast evaluation to finance, although a specific focus is devoted to financial applications. The course has several specific objectives:
- Discuss mathematical/statistical tools for forecast evaluation and model selection for scalar, vector and matrix valued outcomes and lay their theoretical foundation
- Provide clear definition and interpretation of the metrics used to assess forecast accuracy
- Focus on problems arising in forecast evaluation of latent target variables with emphasis on the case of volatility forecasts evaluation
- Analyze recent work in forecast evaluation and model selection based on forecast accuracy and discuss their empirical implementation
Planning
- Measures of forecast accuracy: Notion of empirical ranking. Definition of statistical loss functions (in scalar/vector/matrix spaces), construction and interpretation. Definition of economic loss functions (Minimum variance portfolio, Option pricing accuracy). Regression-based evaluation of predictive accuracy. Observable vs. Latent target variables. (3h)
- Special issues 1: Evaluation of nested models. Estimation sampling schemes (Expanding vs. rolling window). Structural breaks/time varying parameters. Absolute vs. conditional forecasting performance (1.5h)
- Tests of Forecast Comparison – Theory and Hypotheses: Single hypothesis (Diebold-Mariano, West, Clark-McCracken, Giacomini-White, Mincer-Zarnowitz), Multiple hypotheses (Superior Predictive Ability, Model Confidence Set). Simple vs. Composite hypotheses (testing equalities vs. weak inequalities) (5h)
- Special issues 2: Evaluation latent target variables forecasts (Volatility forecast evaluation):
- Introduction to ex-post measures of volatility (semi and non.parametric univariate and multivariate) (realized variance, multi-power variations, realized kernels), related issues : microstructure noise, observation frequency, sampling schemes and prefiltering, seasonality, robustification, jump detection and synchronicity (3h)
- Consistency of forecasts ranking under statistical and economic loss functions. Sufficient and necessary conditions for ranking consistency (4.5h)
- Additional topic (depending on time)
- Predictive density evaluation (Diebold-Gunther-Tay, Corradi-Swanson, Amisano-Giacomini) (1h)
References
Andersen T, Bollerslev T, Diebold F, Labys P. (2003) Modeling and forecasting realized volatility. Econometrica 71, 579-625
Barndorff-Nielsen O, Hansen P, Lunde A, Shephard N. (2008) Designing realized kernels to measure the ex post variation of equity prices in the presence of noise. Econometrica, 76 :1481-1536
Clark, T. and M. McCracken (2015), Nested Forecast Model Comparisons : A New Approach to Testing
Equal Accuracy, J. of Econometrics, Vol. 186, 160-177
Clark, T. and M. McCracken (2005), The Power of Tests of Predictive Ability in the Presence of Structural Breaks, J. of Econometrics 124(1), 1-31
Diebold. F, Mariano R. (1995), Comparing Predictive Accuracy, J. of Business and Economic Statistics 13
Elliott G., Timmermann A, (2008). Economic forecasting. J. of Economic Literature 46, 3-56.
Giacomini, R. and H. White (2006), Tests of Conditional Predictive Ability, Econometrica 74(6).
Hansen P. (2006) A test for superior predictive ability, J. Business and Economic Statistics, 23 :365-380.
Hansen P, Lunde A, Nason J. (2011) The model confidence set, Econometrica, 79 :453-497.
Hansen P, Lunde A. (2006), Consistent ranking of volatility models. J. of Econometrics, 131 :97-121
Laurent S, Rombouts JVK, Violante F. (2013), On loss functions and ranking forecasting performances of multivariate volatility models, J. of Econometrics 173 1-10
Mincer, J. and V. Zarnowitz (1969), The Evaluation of Economic Forecasts, in J. Mincer, ed., Economic
Forecasts and Expectations, 3-46
Patton A. (2011), Volatility forecast comparison using imperfect volatility proxies. J. of Econometrics, 160
West K, McCracken M. (1998), Regression-Based Tests of Predictive Ability, Int. Economic Review, 39
White H. (2000) Reality check for data snooping. Econometrica, 68 :1097-1126
Corradi, V. and N. Swanson (2001) Predictive Density Evaluation, in : Granger, C., G. Elliott and A.
Timmermann (eds.), Handbook of Economic Forecasting Vol. 1
Amisano, G. and R. Giacomini (2007), Comparing Density Forecasts via Weighted Likelihood Ratio Tests, J. of Business and Economic Statistics 25, 177-190