Advanced Machine Learning


Objective

The objective of this course is to present the theoretical basics of statistical learning by focusing essentially on binary classification.

The statistical (and algorithmic) complexity of this problem will be considered through the analysis of the Empirical Risk Minimization first, and then other general class of algorithms : SVM, Neural Nets, Boosting and Random Forest (if time permits). Their statistical properties will be discussed and compared.

These algorithms will be applied to real data during the TP sessions.

Planning

  1. The framework and examples
  2. Empirical Risk Minimization
  3. VC dimension and inequality
  4. Convexification and general losses
  5. Chaining
  6. Support Vector Machines
  7. Boosting
  8. Feed Forward Neural Nets

Références

 

Y. Mansour, Machine Learning: Foundations, Tel-Aviv University, 2013
P. Rigollet, Mathematics of Machine Learning, MIT, 2015
A. Ng, Machine Learning, Stanford, 2015
S. Kakade and A. Tewari, Topics in Artificial Intelligence, TTIC, 2008
General Books
Luc Devroye, Laszlo Gyorfi and Gabor Lugosi. A Probabilistic Theory of Pattern Recognition. Springer 1996.
Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer 2009.
Williams, Graham. Data Mining with Rattle and R : The Art of Excavating Data for Knowledge Discovery Springer 2011.
Luis Torgo. Data Mining with R: Learning with Case Studies. Chapman & Hall/CRC Data Mining and Knowledge Discovery Series 2010.