Computer Vision - CSCI-UA.0480-001

Semester: Spring 2012.

Time and Location: Monday and Wednesdays 11:00-12:15pm, Room 1221, 719 Broadway.

Instructor: Rob Fergus     

Office hours: Wednesday 12:15-1:15pm, Room 1226, 12th floor, 719 Broadway.

TA: Chaitanya Rudra (cr1512@nyu.edu).

Overview

Computer Vision aims to extract descriptions of the world from pictures or video. In recent years, much progress has been made on this challenging problem. The course will start by looking the established area of a geometric vision. It will then move onto mid-level problems such as tracking and segmentation. The final part of the course will focus on recognition, particularly on the problem of detecting object classes (e.g. bottles, shoes, cars) in images, currently a topic much reserach interest.

Prerequisites

The course will be suitable for advanced undergraduates. A reasonable knowledge of linear algebra will be required, along with some basic concepts in machine learning. The homeworks will require Matlab, so familiarity with it is desirable, although not essential.

Assessment

Assessment will be through four graded homework assignments (total of 70%) plus a final exam (30%).

Late Policy

The policy regarding late homework is simple: if it is late, you will get zero marks.

Schedule

Date Time Topics Relevant Book Chapters
WEEK 1
Mon 01/23/2012
 11:00-12:15 1. Introduction, Image Formation Pt. 1 Szeliski, Ch. 1 and 2; F & P, Ch. 1
Wed 01/25/2012
 11:00-12:15 2. Image Formation Pt. 2 (Slides - PPT) (Slides - PDF)
WEEK 2
Mon 01/30/2012
 11:00-12:15 3. Filtering & Edges (Slides - PPT) (Slides - PDF) Szeliski, Ch. 3 and 4; F & P, Ch. 6, 7 and 8
Wed 02/01/2012
 11:00-12:15 4. Lighting, Color (Slides - PPT) (Slides - PDF)
WEEK 3
Mon 02/06/2012
 11:00-12:15 5. Corner & Region detection. Szeliski, Ch. 3 and 4; F and P, ch. 3 and 16; Lowe 2004
Wed 02/08/2012
 11:00-12:15 6. Region representation. (Slides - PPT) (Slides - PDF)

Assignment 1 Out (PDF) (assignment1.zip)
WEEK 4
Wed 02/13/2012
 11:00-12:15 7. Fitting, RANSAC Szeliski, Ch. 6; F & P sec. 3.1, ch. 15; Winder and Brown 2007
Wed 02/15/2012
 11:00-12:15 8. Image Alignment, Optical Flow
WEEK 5
Mon 02/20/2012
 President's Day Holiday
Wed 02/22/2012
 11:00-12:15 9. Epipolar geometry Szeliski, Ch. 7; H and Z, ch. 9-12; F and P, ch. 10 and 11
WEEK 6
Mon 02/27/2012
 11:00-12:15 10. Stereo reconstruction Szeliski, Ch. 7; H and Z, ch. 9-12; F and P, ch. 10 and 11
Wed 02/29/2012
 11:00-12:15 11. Multiview Stereo, Structure from Motion
Wed 02/29/2012
 11:00 Assignment 1 Due

Assignment 2 Out
WEEK 7
Mon 03/05/2012
 11:00-12:15 12. Structure from Motion
Wed 03/07/2012
 11:00-12:15 13. Introduction to Recognition.
WEEK 8
Spring Break (No class)
WEEK 9
Mon 03/19/2012
 11:00-12:15 14. Specific Object Recognition Szeliski, Ch. 14.
Wed 03/21/2012
 11:00-12:15 15. Faces
WEEK 10
Mon 03/26/2012
 11:00-12:15 16. Recognition - Bag of words models Pt. 1 Szeliski, Ch. 14.
Wed 03/28/2012
 11:00-12:15 17. Recognition - Bag of words models Pt. 2
Wed 03/28/2012
 11:00 Assignment 2 Due

Assignment 3 Out
WEEK 11
Mon 04/02/2012
 11:00-12:15 18. Recognition - Discriminative models Pt. 1
Wed 04/04/2012
 11:00-12:15 19. Recognition - Discriminative models Pt. 2
WEEK 12
Mon 04/09/2012
 11:00-12:15 20. Parts-based models
Wed 04/11/2012
 11:00-12:15 21. Segmentation Szeliski, Ch. 5
WEEK 13
Mon 04/16/2012
 11:00-12:15 22. Context
Wed 04/18/2012
 11:00-12:15 23. Activity Recognition
Wed 04/18/2012
 11:00 Assignment 3 Due

Assignment 4 Out (Shi and Malik, PAMI 2000)
WEEK 14
Mon 04/23/2012
 11:00-12:15 24. Attributes
Wed 04/25/2012
 11:00-12:15 25. Hierarichal Models Pt. 1
WEEK 15
Mon 04/30/2012
 11:00-12:15 26. Hierarichal Models Pt. 2
Wed 05/02/2012
 11:00-12:15 27. Internet Vision
EXAM WEEK
Wed 05/09/2012
 11:00 Assignment 4 Due (Note that this is a strict deadline)

Acknowledgments

The instructor would like to thank Andrew Zisserman and Svetlana Lazebnik for making their slides available. Thanks also go to Fei-Fei Li and Antonio Torralba for creating the ICCV'05/CVPR'07 object recognition tutorial slides used in classes 11,12,13.

Textbook

The main text book that we will use is:

Szeliski, Richard, Computer Vision: Algorithms and Applications Springer, 2011. This book is available in electronic form at: Link

There are also a couple of other text books relevant to the course, although we won't be directly using them:

Forsyth, David A., and Ponce, J. Computer Vision: A Modern Approach, Prentice Hall, 2003.

Hartley, R. and Zisserman, A. Multiple View Geometry in Computer Vision, Academic Press, 2002.

Both these are available from the CIMS library.

For the object recognition part of the course, please see the Object Reconition Short Course. Link

Additional Material

Matlab guides

Matlab tutorial by Hany Farid and Eero Simoncelli Link

A more comprehensive Matlab tutorial by David Griffiths Link

Further documentation on Matlab can be found here Link

Books

Palmer, Stephen E. Vision Science: Photos to Phenomenology, MIT Press, 1999.

Strang, Gilbert. Linear Algebra and Its Applications 2/e, Academic Press, 1980.

Wandell, Brian A. Foundations of Vision, Sinauer, 1995.