Computer Vision - CSCI-GA.2271-001

Semester: Fall 2022.

Time and Location: Thursday 7:10-9:00pm EST, 19 University Place, Room 102 Virtual Link .

Instructor: Rob Fergus     

Office hours: Thursday 9.15pm, 19 University Place, room 102.

Course Tutors:
Nikhil Verma (nv2099) -- Office hours: Wed 3-4pm
Jahnvi Arya (ja4158) -- Office hours: Thurs 9-10am
Shravan Chaudhari (ssc10020) -- Office hours: Fri 3-4pm
Tutor zoom link

Course Graders:
Kranthi Kiran GV (kg2769)
Harshita Kukreja (kh3203)
Ayush Jain (aj3152)

Piazza Link: here


Overview

Computer Vision aims to extract descriptions of the world from pictures or video. In recent years, much progress has been made on this challenging problem. The course will start by looking the established area of geometric vision. It will then introduce the convolutional network model and describe the profound impact that it has had on problems in recognition, segmentation and many other vision problems.

Prerequisites

The course will be suitable for master's students and advanced undergraduates. A reasonable knowledge of linear algebra will be required, along with some basic concepts in machine learning. The homeworks will require PyTorch, so familiarity with it is desirable, although not essential.

Assessment

Assessment will be through three graded homework assignments and a final course project. The weighting across these will be: 17% 17%, 17%, 49%. Directions for the project (final report).

Late Policy

The policy regarding late homework is as follows: (a) assignments that are late by less than 24hrs will suffer a 10% reduction; (b) those between 24 and 72 hrs late will suffer a 25% reduction and (c) those more 72hrs late will suffer a 50% reduction. You are strongly encouraged to start the assignment early and don't be afraid to ask for help.

Schedule

Date Time Topics Relevant Book Chapters
WEEK 1
Thur 09/01/2022
 19:10-21:00 1. Introduction, Image Formation Pt. 1     **REMOTE CLASS** Link Szeliski, Ch. 1 and 2; F & P, Ch. 1
Thur 09/01/2022
 19:10-21:00 2. Image Formation Pt. 2 (Slides - PDF)   **REMOTE CLASS**
WEEK 2
Thur 09/08/2022
 19:10-21:00 3. Neural nets 1
Thur 09/08/2022
 19:10-21:00 4. Neural nets 2 (Slides - PDF)

Assignment 1 Out (Part 1) (Part 2) (Dataset)
WEEK 3
Thur 09/15/2022
 19:10-21:00 5. Convolutional Networks 1 (Slides - PDF)
Thur 09/15/2022
 19:10-21:00 6. Convolutional Networks 2
WEEK 4
Thur 09/22/2022
 19:10-21:00 7. Object Classification (Slides - PDF)
Thur 09/22/2022
 19:10-21:00 8. Object Detection Pt.1 (PDF)
WEEK 5
Thur 09/29/2022
 19:10-21:00 9. Transformers in vision (Slides - PDF)
Thur 09/29/2022
 19:10-21:00 10. Perceivers in vision and beyond
Thur 09/29/2022
 23:59 Assignment 1 Due

Assignment 2 Out Link
WEEK 6
Thur 10/06/2022
 19:10-21:00 11. Semantic segmentation & Image processing (PDF)
Thur 10/06/2022
 19:10-21:00 12. Efficient evaluation (PDF)
WEEK 7
Thur 10/13/2022
 19:10-21:00 13. Self-supervised learning [Guest lecture: Ishan Misra, FAIR]
Thur 10/13/2022
 19:10-21:00 14. Self-supervised learning
Thur 10/13/2022
 23:59 Project Abstracts Due
WEEK 8
Thur 10/20/2022
 19:10-21:00 15. Unsupervised learning & GANs (PDF)
Thur 10/20/2022
 19:10-21:00 16. Unsupervised learning & GANs
WEEK 9
Thur 10/27/2022
 19:10-21:00 17. RNNs & Image Captioning (PDF)
Thur 10/27/2022
19:10-21:00 18. Text-to-Image models (PDF)
Thur 10/27/2022
 23:59 Assignment 2 Due
WEEK 10
Thur 11/03/2022
 19:10-21:00 19. Visual Q & A (PDF)
Thur 11/03/2022
 19:10-21:00 20. Visual Q & A
WEEK 11
Thur 11/10/2022
 19:10-21:00 21. Optical flow
Thur 11/10/2022
 19:10-21:00 22. Video Recognition (PDF)
WEEK 12
Thur 11/17/2022
 19:10-21:00 23. Local Features, Image Alignment and Matching Szeliski, Ch. 6; F and P, ch. 3.1 and 15; Lowe 2004
Thur 11/17/2020
 19:10-21:00 24. Local Features, Image Alignment and Matching (PDF) Winder and Brown 2007

Assignment 3 Out (Colab) (zip)
WEEK 13
Thanksgiving Break (No class)
WEEK 14
Thur 12/01/2022
 19:10-21:00 25. Epipolar geometry Szeliski, Ch. 7; F & P ch. 10 & 11;
Thur 12/01/2022
 19:10-21:00 26. Stereo reconstruction (PDF)
WEEK 15
Thur 12/08/2022
 19:10-21:00 27. NERFs Szeliski, Ch. 14
Thur 12/08/2022
 19:10-21:00 28. Large scale structure from motion. (Slides - PDF)
EXAM WEEK
Thur 12/15/2022
 19:10-21:00 Project Presentations
Thur 12/15/2022
 23:59 Assignment 3 Due
Thur 12/15/2022
 23:59 Project Due (Note that this is a strict deadline)

Acknowledgments

The instructor would like to thank Emily Denton, Kaiming He, Svetlana Lazebnik, Laurens van der Maaten, Yaniv Taigman and Andrew Zisserman for making their slides available.

Textbook

We will not be using any textbook in this class.