 | |
|---|---|
| |
| |
| |
| |
 |  |
 | |
Machine Learning and Pattern Recognition: Schedule
[ Course Homepage | Schedule and Course Material | Mailing List ]
|
|
|
||||||||||||||
|
This page contains the schedule, slide from the lectures, lecture notes, reading lists, assigments, and web links. I urge you to download the DjVu viewer and view the DjVu version of the documents below. They display faster, are higher quality, and have generally smaller file sizes than the PS and PDF. Full-text search is provided for the entire collection of slides and papers. Click here to search You can have a look at the schedule and class material for the version of this course taught during the Spring 2004 semester, but be warned that the new edition is significantly different.
Subjects treated: Intro, types of learning, nearest neighbor, how biology does it, linear classifier, perceptron learning procedure, linear regression, Recommended Reading:
Subjects treated: Energy-based models, minimum-energy machines, loss functions. Linear machines: perceptron, logistic regression. Linearly parameterized classifiers: Polynomial classifiers, basis function expansion, RBFs, Kernel-based expansion.
Subjects treated: Multi-Module learning machines. Vector modules and switches. Multilayer neural nets. Backpropagation Learning. Intro to Model Selection, structural risk minimization, regularization. Slides on Regularization: [DjVu | PDF | PS] Slides on Multi-Module Back-Propagation: [DjVu | PDF | PS] Required Reading: Gradient-based Learning Applied to Document Recognition by LeCun, Bottou, Bengio, and Haffner; pages 1 to the first column of page 18: [DjVu | .ps.gz ] Homework Assignements: Linear Classifier: implementing the Perceptron Algorithm, MSE Classifier (linear regression), Logistic Regression. Details and datasets below:
Subjects treated: Trainers; complex topologies; special modules; Cross-entropy and KL-divergence; RBF-nets, Mixtures of Experts; Parameter space transforms; weight sharing; convolution module; TDNN; Recurrent nets. Homework Assignements: Computing Jacobians: pen-and-paper homework to get familiar with computing jacobians. Click on one of these links to get the text of the homework: [DjVu | PDF | PS] Due Date is Tuesday October 18th, before the lecture.
Subjects treated: Convolutional Networks; Image recognition, object detection, and other applications; Slides: talk on object recognition with convolutional nets: [DjVu| PDF] Required Reading: If you haven't read it already: Gradient-based Learning Applied to Document Recognition by LeCun, Bottou, Bengio, and Haffner; pages 1 to the first column of page 18: [ DjVu | .ps.gz ] Optional Reading: Fu-Jie Huang, Yann LeCun, Leon Bottou: "Learning Methods for Generic Object Recognition with Invariance to Pose and Lighting.", Proc. CVPR 2004. [DJVU, PDF, PS.GZ]
Slides:: same as last week.
Subjects treated: Refresher probability theory; Bayesian Estimation, Maximum Likelihood Estimation, Maximum A Posteriori Estimation, Negative Log-Likelihood Loss Functions. Slides: Refresher on Probability Theory: [DjVu | PDF | PS] Slides: Bayesian Learning: [DjVu | PDF | PS] Required Reading:
Subjects treated: Ensemble methods, More on Bayesian Learning, Bagging, Boosting. Learning Theory, Bounds, VC-Dimension. Slides: Ensemble Methods: [DjVu | PDF | PS]
Subjects treated: Optimization: Convergence of gradient-based optimization and acceleration techniques. Gauss-Mewton, Levenberg-Marquardt, BFGS, Conjugate Gradient. Slides: Required Reading: Efficient Backprop, by LeCun, Bottou, Orr, and Muller: [ DjVu | .ps.gz ]
Subjects treated: Unsupervised Learning: Clustering: K-Means; Principal Component Analysis, Auto-Encoders. Density Estimation: Parzen Windows. Gaussian Density Estimation. Latent variables. Unsupervised learning: [DjVu | PDF | PS]
Click on this links to get the homework: hw-backprop.tgz. Due Date is Tuesday November 22th, before the lecture.
note: the guest lecture is cancelled.. the Estimation-Maximization algorithm. Mixtures of Gaussians. Unsupervised learning: [DjVu | PDF | PS]
Subjects treated: Modeling distributions over sequences. Learning machines that manipulate graphs. Finite-state transducers. Graph Transformer Networks. Introduction to Hidden Markov Models (HMM). Required Reading:
Subjects treated: Intro to graphical models, Belief Networks and Factor Graphs, Inference, Belief Propagation, Boltzmann Machines. Suggested Reading: David Mackay's book Information Theory, Inference, and Learning Algorithms. (available for free download in PDF and DjVu). Homework Assignements: K-Means and Mixture of Gaussians Model: Click on this links to get the homework: hw-unsup.tgz. Due Date is Friday December 16th.
|
|
||||||||||||||
|
|
|