CSC2515 Fall 2003 - Weekly and Other Readings

The textbook for the class is Hastie, Tibshirani, Friedman Elements of Statistical Learning.

I will also be handing out class notes as we go along.

Some classic papers will be assigned as weekly readings.

We will also be covering material similar to a variety of chapters from a few other books which I will point out in class.

September 10
L.G. Valiant, A Theory of the Learnable [pdf, 9pages]
September 16 (choose one)
R.A. Fisher, The Use of Multiple Measurements in Taxonomic Problems [pdf, 10pages] or
J.R. Quinlan, Induction of Decision Trees [pdf, 26pages]
September 23
Michael Jordan, Why the logistic function? [pdf , ps.gz, 13pages]
September 30
Robert Tibshirani Regression shrinkage and selection via the lasso [pdf , ps.gz, 28pages]
Oct 7
Press et al. Numerical Recipes in C chapters 10.0 - 10.7 inclusive [online pdf]
October 14
Michael I. Jordan and Robert A. Jacobs (1994), Hierarchical Mixtures of Experts and the EM Algorithm [pdf , ps.gz, 36pages]
Note: only read pages 1-10 this week.
October 21
rest of the Mixtures Experts article
October 28
Geoff Hinton and Radford Neal, A View of the EM Algorithm, Learning in Graphical Models (1998), [pdf , ps.gz, 14pages]
November 4 and November 11
Sam Roweis and Zoubin Ghahramani, A Unifying Review of Linear Gaussian Models, Neural Compuation (1999), [pdf 41pages]
Skip sections 5.4, 6.2, 8, 9, A.3
November 11
Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. [pdf7pages]
November 18
Golub, Heath, Wahba, Generalized Cross Validation, Technometrics 1979. [in class, 9 pages]
November 25
Rob Shapire, The Strength of Weak Learnability., Machine Learning 1990. [pdf31pages]
December 2
Corinna Cortes and Vladimir Vapnik, Support Vector Networks, Machine Learning 20(3): 273-297 (1995) [ps.gz31pages]

Probability and Statistics Review [ps.gz].
A tutorial by Andrew Moore at CMU on Bayesian methods.
Another tutorial by Andrew Moore on probabilities for machine learning and data mining.
Some notes from Geoff Gordon at CMU on SVMs and kernels.
A short MATLAB tutorial.

Linear Algebra, v1.3 [ps.gz].

Rob Shapire, Boosting Overview [ps.gz, 23 pages]
Leo Brieman, Bagging Predictors [pdf, 20 pages]
David Wolpert, Stacked Generalization [ps.gz, 57 pages]
David Mackay, Maximum Likelihood and Covariant Algorithms for ICA [ps.gz, 15 pages]
Zoubin Ghahramani and Geoff Hinton, The EM algorithm for Mixtures of Factor Analyzers [ps.gz, 8 pages]
C.K. Chow, C.N. Liu, Approximating Discrete Probability Distributions with Dependence Trees [pdf, 6 pages]
Marina Meila, An accelerated Chow and Liu algorithm [ps.gz, 12 pages]
Kwong Yung, Explaining the Stein Paradox [pdf, 5 pages]
S. Gallant. Perceptron-based learning algorithms. [pdf, 13pages]
Yoav Freund, Rob Schapire, Large Margin Classification Using the Perceptron Algorithm [ps.gz, 19pages]
Pedro Domingos, Michael Pazzani, On the Optimality of the Simple Bayesian Classifier [pdf, 28pages]