VLG Group
Group Meetings
Y. LeCun's website
CS at Courant
Courant Institute

Introduction to Machine Learning and Pattern Recognition: Schedule

[ Course Homepage | Schedule and Course Material | Mailing List ]

This page contains the schedule, slide from the lectures, lecture notes, reading lists, assigments, and web links.

I urge you to download the DjVu viewer and view the DjVu version of the documents below. They display faster, are higher quality, and have generally smaller file sizes than the PS and PDF.

01/17: Introduction and basic concepts

Subjects treated: Intro, types of learning, nearest neighbor, how biology does it, linear classifier, perceptron learning procedure, linear regression,

Slides: [DjVu | PDF | PS]

01/22: Introduction to Lush, Computer Lab

IMPORTANT NOTE: This session will start in the usual classroom WWH-102, but the second part will be in room WWH-512 (Sun computer lab on the 5-th floor).

Subjects treated: An introduction to the Lush programming language. Implementing the perceptron learning algorithm.

The instructor for this lab seesion will be Fu-Jie Huang (Jie).

You should all have a login on the Sun machines in the lab.

If you have laptop, you are welcome to install Lush on it and to use it for the computer lab sessions instead of the labs's Sun workstations. Lush runs on Linux, Mac OS-X, Solaris, and on Windows under Cygwin.

Click here to get the files for the lab and the homework.

HOMEWORK 01b: The homework is due 01/31 before the class (NOTE: new due date).

Slides: [DjVu | PDF | PS]

Lush Tutorial: read the short tutorial on Lush at the Lush documentation page.

01/24: Perceptron, Linear Regression

Subjects treated: Intro, perceptron convergence theorem, multivariate calculus refresher, loss functions, linear regression, LMS/Adaline.

Slides: [DjVu | PDF | PS]

01/29: Energy-Based Models, Loss Functions, Linear Regression

Subjects treated: Energy-Based Models, Inference, Loss Functions, Logistic Regression.

Slides: [DjVu | PDF | PS]

01/31: Lab: Linear Classifiers

IMPORTANT NOTE: Location to be announced. .

Subjects treated: LAB: perceptron, LMS, Logistic Regression.

HOMEWORK 02a: Click here to get the files for the lab and the homework.

To uncompress the homework file on Unix/Linux do "tar xvfz lab-02a.tgz". This will create a directory named "lab-02a". On Windows, use Winzip.

The homework is due 02/14 before the class.

02/05: Generalization, Regularization

Subjects treated: Learning and Generalization, Regularization.

Slides: [DjVu | PDF | PS]

02/07: Multi-Module Systems, Backpropagation

Subjects treated: Multilayer and Multi-Module Systems, Gradient Back-Propagation.

Slides: [DjVu | PDF | PS]

HOMEWORK 05b: computing Jacobians. Click one these links to get the text of the homework: [DjVu | PDF | PS]

The homework is due 02/14 before the class.

02/12: Modules and Architectures

Subjects treated: Modules and Architectures.

Slides: [DjVu | PDF | PS]

02/14: Special Architectures

Subjects treated: Special Architectures for Time Series, Images, Audio, Video.

Slides: [DjVu | PDF | PS]

HOMEWORK 09a: speech recognition with backpropagation and multilayer nets. Click here to get the text of the homework (caution: the file is 11MB).

The homework is due 02/28 before the class.

02/19: President's Day: NO CLASS

02/21: Audio and Image Classification

Subjects treated: Convolutional Networks for speech and image classification.

Slides: object recognition [DjVu].

Face detection, and other applications

Slides: face detection [DjVu].

Required Reading:

Gradient-based Learning Applied to Document Recognition by LeCun, Bottou, Bengio, and Haffner; pages 1 to the first column of page 18: [DjVu | .ps.gz ]

02/26: a primer on probability theory

Subjects treated: distributions, marginalization, joint, conditional, exponential family, Gaussians.

Slides: [DjVu | PDF | PS]

02/28: intro to unsupervised learning

Subjects treated: density estimation. Maximum likelihood estimation, Gaussian estimation, Parzen Windows.

Slides: [DjVu | PDF | PS]

03/05: Dimensionality Reduction

Subjects treated: principal component analysis

Slides: [DjVu | PDF | PS]

HOMEWORK 10b: Principal Component Analysis Click here to get the text of the homework

The homework is due 03/21 before the class.

03/07: Clustering, Data Compression

Subjects treated: K-Means clustering, vector quantization, image compression.

Slides: [DjVu | PDF | PS]

The homework is due 03/21 before the class.

HOMEWORK 11a: K-Means Clustering Click here to get the text of the homework



03/19: Ensemble Methods, Boosting

Subjects treated: Ensemble methods, Jackknife, Bagging, Boosting.

Slides: [DjVu | PDF | PS]

03/21: Support Vector Machine

Support Vector Machines and Kernel Methods

03/26: Expectation-Maximization

Subjects treated: latent variables, Expectation-Maximization algorithm (EM), mixtures of Gaussians.

Slides: [DjVu | PDF | PS]

HOMEWORK 12a: Mixture of Gaussians Click here to get the text of the homework

The homework is due 04/04 before the class.

03/28: Lab: gblearn2 library and convolutional nets

Subjects treated: Training a handwritten digit recognizer.

FINAL PROJECT: most of the proposed topics of final projects will be related to this lab session. You can make a proposal for a different subject outside of the proposed list if you want. Proposed subjects will include various ways of training various architectures on the MNIST dataset of handwritten digits. Students will have 15 minutes to present the results of their project to the class in an oral exam session on Thursday, May 5 from 4:00 PM to 5:30 PM. You can do the project by yourself or in groups of two.

04/02: Fast Optimization Methods

Subjects treated: Optimization, Hessian, Convergence of gradient descent, Newton's algorithm, Levenberg-Marquardt, Conjugate Gradient.

Required reading: "Efficient Backprop" [DjVu] [PS.GZ] [PDF]

04/04: Bayesian Learning

Subjects treated: Bayesian Learning Methods.

Slides: [DjVu | PDF | PS]

04/09: Distributions on Strings

Subjects treated: Distributions on strings, weighted finite-state machines, finite-state transducers.

Slides: Tutorial by Mehryar Mohri and Michael Riley: [DjVu | PDF | PS]

04/11: Introduction to Graphical Models

Subjects treated: Graphical models, factorized probability distributions, belief propagation and the sum-product algorithm.

04/16: Learning with Sequences

Subjects treated: Graphs Transformer Networks, sequence labeling, discriminative training.

Required Reading:

Gradient-based Learning Applied to Document Recognition by LeCun, Bottou, Bengio, and Haffner; pages 18 (sectionC) to 40. [DjVu | .ps.gz ]

04/18: Applications

Applications of Machine Learning.

04/23: Review, Open Questions

Review, Open Questions


Project Presentations.


Project Presentations.