Administrative Information
- Time/Place: Mondays 12:30-3:00 PM in CIWW 317
- Instructor: Marshall Ball, Office Hours: TBD
/p>
In the late 1940s, Claude Shannon introduced a mathematical theory of information, quantifying information in terms of uncertainty. Since Shannon's pioneering work on the limits of data compression and reliable/secure communication, intuitions and techniques from information theory have impacted not just our modern communication infrastructure but also a diverse array of other scientific endeavors, including theoretical computer science. In this course, we will begin by covering the foundations of information theory (entropy, mutual information, KL-divergence, etc) before branching off to explore various applications, primarily in theoretical computer science. Potential application topics include: channel and source coding, error correcting codes, communication complexity, hardness amplification, data structures, Kolmogorov complexity, information-theoretic cryptography, pseudoentropy, the Lovasz Local Lemma, and applications in combinatorics. While there are no specific prerequisites, fluency in basic probability and mathematical maturity are required.
Topics will draw from the following:
4 Problem Sets
The primary prerequisite is mathematical maturity. You should be comfortable reading and writing proofs. Some familiarity with the basics of algorithms, the theory of computation, and probability is expected.
If you are unsure about whether this class is suitable for you, please contact the instructor via email.
While we will not be following a textbook, a number of excellent resouces are available.
| Class | Date | Topic | Notes |
|---|---|---|---|
| 1 | Sep 14 | Introduction: Entropy, Conditional Entropy, Mutual Information, KL Divergence | |
| 2 | Sep 21 | ||
| 3 | Sep 28 | ||
| 4 | Oct 5 | ||
| 5 | Oct 12 | ||
| 6 | Oct 19 | ||
| 7 | Oct 26 | ||
| 8 | Nov 2 | ||
| 9 | Nov 9 | ||
| 10 | Nov 16 | ||
| 11 | Nov 23 | ||
| 12 | Nov 30 | ||
| 13 | Dec 7 | ||
| 14 | Dec 14 |
Homework should be submitted in PDF form in Gradescope. We prefer homework submissions typeset in LaTex. If you are not familiar with LaTex, it is a great skill to learn. Overleaf provides a simple web interface for writing and compiling LaTex (as well as extensive documentation). We will provide LaTex source for you to edit. You are encouraged to insert scanned figures or illustrations where appropriate. Scanned handwritten submissions will only be graded if perfectly legible. If you are unsure about your handwriting, I strongly suggest you type your solutions.
An important part of this class is about learning to communicate your mathematical ideas and proofs clearly and concisely. Accordingly, you will be graded not simply for correctness, but also clarity.
We strongly encourage you to discuss assignments with up to 3 peers, but you must (a) list the names of your discussion partners on your submission, and (b) you must write up your solution on your own. You may not look at the written solutions of any other student before submitting your own solution. If you do not not list the names of your collaborators, you will be penalized.
Late homework will not be accepted, but the lowest scored homework will be dropped.
You must explicitly acknowledge any external resources consulted in your homework.
However, you are not allowed to consult any resource for the purpose of finding homework solutions. For example, you may not consult homework solutions for a previous version of this class or ask an LLMViolations to this policy is plagiarism and will not be tolerated.
Your work should be your own. Students are should be aware of the CS Department's Policy on Academic Integrity. Violations of academic integrity will not be tolerated.
As a nonsectarian, inclusive institution, NYU policy permits members of any religious group to absent themselves from classes without penalty when required for compliance with their religious obligations. The policy and principles to be followed by students and faculty may be found in the University Calendar Policy on Religious Holidays.
Academic accommodations are available to any student with a chronic, psychological, visual, mobility, learning disability, or who is deaf or hard of hearing. Students should please register with the Moses Center for Students with Disabilities.