CSC2515 Fall 2002 - Lectures

Tentative Lecture Schedule

  • Sept 11
    Lecture 1: Introduction to Machine Learning, Generalization and Capacity (notes)

  • Sept 18 --Classification 1:
    KNN, linear discriminants, decision trees (notes)

  • Sept 25 -- Classification 2:
    naive Bayes, logistic regression, neural nets for classification (notes)

  • Sept. 30: Assignment 1 (Classification) posted

  • Oct 2 -- Regression 1:
    constant model, linear models, generalized additive models (e.g. RBFs), locally weighted regression, multilayer perceptrons/neural networks (notes)

  • Oct 9 -- Regression 2 and Basic Optimization:
    error surfaces, weight space, credit assignment problem, neural networks, kolmogorov's theorem, backprop algorithm for efficiently computing gradients, gradient descent, stochastic gradient, (notes)

  • October 15: Assignment 2 (Regression) handed out

  • October 16: Assignment 1 due at the start of class

  • Oct 16 -- Supervised Mixtures and Advanced Optimization:
    conjugate gradient, bound optimization , convexity, enforcing constraints mixtures of experts, piecewise linear models (notes)

  • Oct 23 -- Unsupervised Learning 1: Mixture models and the EM Algorithm:
    missing data, hidden variables, Jensen's inequality, lower bound on marginal likelihood, free energy interpretation, inference, (notes)

  • Oct30 -- Unsupervised Learning 2: Trees & Clustering
    K-means, heirarchical clustering, soft competitive learning (Mixtures of Gaussians), maximum likelihood trees, optimal tree structure (notes)

  • November 6: Assignment 2 due

  • November 6: Assignment 3 posted

  • November 6 -- Unsupervised Learning 3:
    Continuous latent variable models, Factor Analysis, (Probabilistic) PCA, Mixtures of Factor Analyzers, Independent Components Analysis (notes)

  • Nov 13 -- Time Series Models
    autoregressive/Markov models, hidden Markov models (notes)

  • Nov 20: Assignment 3 due at the start of class

  • Nov 20 -- Capacity Control:
    generalization and overfitting, No free lunch theorems, high dimensional issues.
    capacity control methods: weight decay, early stopping, cross validation, model averaging, intro to Bayesianism (notes)

  • Nov 27 -- Meta-Learning Methods:
    stacking, bagging, boosting (notes)

  • Dec 4 -- Kernel methods:
    the kernel trick, support vector machines, kernel perceptrons, sparsity, capacity control, dual problems (notes)

  • Dec 18 -- projects due

  • Extra topics we didn't have time for
    • other kernel machines: gaussian processes
    • linear dynamical systems, Kalman filtering
    • Approximate inference and learning:
      sampling, variational approximations, loopy belief propagation
    • Spectral Methods:
      Isomap,LLE, spectral clustering,
    • MaxEnt models:
      maximum entropy/energy-based models, iterative scaling, products of experts, dependency nets,
    • Matrix Factorizations:
      aspect models/LDA/plaids, non-negative matrix factorization
    • Automatic Structure Learning:
      sparsity priors, empirical Bayes, automatic relevance determination (MLII), structural EM


[ Home | Course Information | Lecture Schedule/Notes | Textbook/Readings | Assignments/Tests | Computing | ]

CSC2515 - Machine Learning || www.cs.toronto.edu/~roweis/csc2515/