|
CSC2515 Fall 2002 - Lectures
Tentative Lecture Schedule
- Sept 11
Lecture 1: Introduction to Machine Learning, Generalization and Capacity
(notes)
- Sept 18 --Classification 1:
KNN, linear discriminants, decision trees
(notes)
- Sept 25 -- Classification 2:
naive Bayes, logistic regression, neural nets for classification
(notes)
- Sept. 30: Assignment 1 (Classification) posted
- Oct 2 -- Regression 1:
constant model, linear models, generalized additive models
(e.g. RBFs), locally weighted regression,
multilayer perceptrons/neural networks
(notes)
- Oct 9 -- Regression 2 and Basic Optimization:
error surfaces, weight space, credit assignment problem,
neural networks, kolmogorov's theorem,
backprop algorithm for efficiently computing gradients,
gradient descent, stochastic gradient,
(notes)
- October 15: Assignment 2 (Regression) handed out
- October 16: Assignment 1 due at the start of class
- Oct 16 -- Supervised Mixtures and Advanced Optimization:
conjugate gradient, bound optimization , convexity, enforcing constraints
mixtures of experts, piecewise linear models
(notes)
- Oct 23 -- Unsupervised Learning 1:
Mixture models and the EM Algorithm:
missing data, hidden variables,
Jensen's inequality, lower bound on marginal likelihood,
free energy interpretation, inference,
(notes)
- Oct30 -- Unsupervised Learning 2: Trees & Clustering
K-means, heirarchical clustering, soft competitive learning
(Mixtures of Gaussians), maximum likelihood trees, optimal tree structure
(notes)
- November 6: Assignment 2 due
- November 6: Assignment 3 posted
- November 6 -- Unsupervised Learning 3:
Continuous latent variable models, Factor Analysis, (Probabilistic)
PCA, Mixtures of Factor Analyzers, Independent Components Analysis
(notes)
- Nov 13 -- Time Series Models
autoregressive/Markov models, hidden Markov models
(notes)
- Nov 20: Assignment 3 due at the start of class
- Nov 20 -- Capacity Control:
generalization and overfitting, No free lunch theorems,
high dimensional issues.
capacity control methods: weight decay,
early stopping, cross validation, model averaging, intro to Bayesianism
(notes)
- Nov 27 -- Meta-Learning Methods:
stacking, bagging, boosting
(notes)
- Dec 4 -- Kernel methods:
the kernel trick, support vector machines, kernel perceptrons,
sparsity, capacity control, dual problems
(notes)
- Dec 18 -- projects due
- Extra topics we didn't have time for
- other kernel machines: gaussian processes
- linear dynamical systems, Kalman filtering
- Approximate inference and learning:
sampling, variational approximations, loopy belief propagation
- Spectral Methods:
Isomap,LLE, spectral clustering,
- MaxEnt models:
maximum entropy/energy-based models, iterative scaling,
products of experts, dependency nets,
- Matrix Factorizations:
aspect models/LDA/plaids, non-negative matrix factorization
- Automatic Structure Learning:
sparsity priors, empirical Bayes,
automatic relevance determination (MLII), structural EM
[
Home |
Course Information |
Lecture Schedule/Notes |
Textbook/Readings |
Assignments/Tests |
Computing |
]
CSC2515 - Machine Learning || www.cs.toronto.edu/~roweis/csc2515/
|