CSC2515 Fall 2003 - Weekly and Other Readings


The textbook for the class is Hastie, Tibshirani, Friedman Elements of Statistical Learning.

I will also be handing out class notes as we go along.

Some classic papers will be assigned as weekly readings.

We will also be covering material similar to a variety of chapters from a few other books which I will point out in class.

Weekly Readings

  • September 10
    L.G. Valiant, A Theory of the Learnable [pdf, 9pages]

  • September 16 (choose one)
    R.A. Fisher, The Use of Multiple Measurements in Taxonomic Problems [pdf, 10pages] or
    J.R. Quinlan, Induction of Decision Trees [pdf, 26pages]

  • September 23
    Michael Jordan, Why the logistic function? [pdf , ps.gz, 13pages]

  • September 30
    Robert Tibshirani Regression shrinkage and selection via the lasso [pdf , ps.gz, 28pages]

  • Oct 7
    Press et al. Numerical Recipes in C chapters 10.0 - 10.7 inclusive [online pdf]

  • October 14
    Michael I. Jordan and Robert A. Jacobs (1994), Hierarchical Mixtures of Experts and the EM Algorithm [pdf , ps.gz, 36pages]
    Note: only read pages 1-10 this week.
  • October 21
    rest of the Mixtures Experts article

  • October 28
    Geoff Hinton and Radford Neal, A View of the EM Algorithm, Learning in Graphical Models (1998), [pdf , ps.gz, 14pages]

  • November 4 and November 11
    Sam Roweis and Zoubin Ghahramani, A Unifying Review of Linear Gaussian Models, Neural Compuation (1999), [pdf 41pages]
    Skip sections 5.4, 6.2, 8, 9, A.3

  • November 11
    Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. [pdf7pages]

  • November 18
    Golub, Heath, Wahba, Generalized Cross Validation, Technometrics 1979. [in class, 9 pages]

  • November 25
    Rob Shapire, The Strength of Weak Learnability., Machine Learning 1990. [pdf31pages]

  • December 2
    Corinna Cortes and Vladimir Vapnik, Support Vector Networks, Machine Learning 20(3): 273-297 (1995) [ps.gz31pages]

Additional Material

  • Probability and Statistics Review [ps.gz].
  • A tutorial by Andrew Moore at CMU on Bayesian methods.
  • Another tutorial by Andrew Moore on probabilities for machine learning and data mining.
  • Some notes from Geoff Gordon at CMU on SVMs and kernels.
  • A short MATLAB tutorial.

Draft Book Chapters

  • Linear Algebra, v1.3 [ps.gz].

    Extra Papers of Interest

    • Rob Shapire, Boosting Overview [ps.gz, 23 pages]
    • Leo Brieman, Bagging Predictors [pdf, 20 pages]
    • David Wolpert, Stacked Generalization [ps.gz, 57 pages]
    • David Mackay, Maximum Likelihood and Covariant Algorithms for ICA [ps.gz, 15 pages]
    • Zoubin Ghahramani and Geoff Hinton, The EM algorithm for Mixtures of Factor Analyzers [ps.gz, 8 pages]
    • C.K. Chow, C.N. Liu, Approximating Discrete Probability Distributions with Dependence Trees [pdf, 6 pages]
    • Marina Meila, An accelerated Chow and Liu algorithm [ps.gz, 12 pages]
    • Kwong Yung, Explaining the Stein Paradox [pdf, 5 pages]
    • S. Gallant. Perceptron-based learning algorithms. [pdf, 13pages]
    • Yoav Freund, Rob Schapire, Large Margin Classification Using the Perceptron Algorithm [ps.gz, 19pages]
    • Pedro Domingos, Michael Pazzani, On the Optimality of the Simple Bayesian Classifier [pdf, 28pages]

    [ Home | Course Information | Lecture Schedule/Notes | Textbook/Readings | Assignments/Tests | Computing | ]

    CSC2515 - Machine Learning ||