
CSC2515 Fall 2003  Lectures
Tentative Lecture Schedule
 Sept 10  FIRST CLASS ONLY ON A WEDNESDAY
Lecture 1: Introduction to Machine Learning, Generalization and Capacity
(notes [ps]
[pdf])
 Sept 16 Classification 1:
KNN, linear discriminants, decision trees
(notes [ps]
[pdf])
 Sept 23  Classification 2:
naive Bayes, logistic regression, neural nets for classification
(notes [ps]
[pdf])
 Sept 30: Assignment 1 (Classification) posted
 Sept 30  Regression 1:
constant model, linear models, generalized additive models
(e.g. RBFs), locally weighted regression,
multilayer perceptrons/neural networks
(notes [ps]
[pdf])
 Oct 7  Objective Functions and Optimization:
error surfaces, weight space, gradient descent, stochastic gradient,
conjugate gradients, second order methods, convexity, enforcing constraints
(notes [ps]
[pdf])
 Oct14: Assignment 1 due at the start of class
 Oct 14  Regression 2 and Supervised Mixtures:
credit assignment problem, neural networks, radial basis networks,
kolmogorov's theorem,
backprop algorithm for efficiently computing gradients,
mixtures of experts, piecewise linear models
(notes [ps]
[pdf])
 Oct21: Assignment 2 (Regression) posted
 Oct 21  Unsupervised Learning 1:
Trees & Clustering
Kmeans, heirarchical clustering, soft competitive learning
(Mixtures of Gaussians), maximum likelihood trees, optimal tree structure
(notes [ps]
[pdf])
 Oct28  Unsupervised Learning 2:
Mixture models and the EM Algorithm:
missing data, hidden variables,
Jensen's inequality, lower bound on marginal likelihood,
free energy interpretation, inference,
(notes [ps]
[pdf])
 Nov 4: Assignment 2 due
 November 4  Unsupervised Learning 3:
Continuous latent variable models, Factor Analysis, (Probabilistic)
PCA, Mixtures of Factor Analyzers, Independent Components Analysis
(notes [ps]
[pdf])
 Nov 11: Assignment 3 posted
 Nov 11  Time Series Models
autoregressive/Markov models, hidden Markov models
(notes [ps]
[pdf])
 Nov 18  Capacity Control:
generalization and overfitting, No free lunch theorems,
high dimensional issues.
capacity control methods: weight decay,
early stopping, cross validation, model averaging, intro to Bayesianism
(notes [ps]
[pdf])
 Nov 25: Assignment 3 due at the start of class
 Nov 25  MetaLearning Methods:
stacking, bagging, boosting
(notes [ps]
[pdf])
 Dec 2  Kernel methods:
the kernel trick, support vector machines, kernel perceptrons,
sparsity, capacity control, dual problems
(notes [ps]
[pdf])
 December 19  projects due
 Extra topics we may or may not have time for
 other kernel machines: gaussian processes
 linear dynamical systems, Kalman filtering
 Approximate inference and learning:
sampling, variational approximations, loopy belief propagation
 Spectral Methods:
Isomap,LLE, spectral clustering,
 MaxEnt models:
maximum entropy/energybased models, iterative scaling,
products of experts, dependency nets,
 Matrix Factorizations:
aspect models/LDA/plaids, nonnegative matrix factorization
 Automatic Structure Learning:
sparsity priors, empirical Bayes,
automatic relevance determination (MLII), structural EM
[
Home 
Course Information 
Lecture Schedule/Notes 
Textbook/Readings 
Assignments/Tests 
Computing 
]
CSC2515  Machine Learning  www.cs.toronto.edu/~roweis/csc2515/
