Advanced Topics in Numerical Analysis: NUMERICAL OPTIMIZATION

Computer Science CSCI-GA.2945-001

Mathematics MATH-GA.2011-002

New York University
Fall Semester 2014

Class meetings: Tuesday, 5:10-7:00pm
Warren Weaver Hall (CIWW), Room 512

Instructor: Margaret Wright,

Office: Warren Weaver Hall (CIWW), Room 430

Office Hours: Monday 3:00-4:50pm, or by appointment

Description of Course Content

Many problems in science, engineering, medicine, and business involve optimization, in which we seek to optimize a mathematical measure of goodness subject to constraints. This course will survey widely used methods for continuous optimization, focusing on both theoretical foundations and implementation as software, with particular attention to numerical issues. Topics include linear programming (optimization of a linear function subject to linear constraints), line search and trust region methods for unconstrained optimization, and a selection of approaches (including active-set, sequential quadratic programming, and interior methods) for constrained optimization.

Textbook (recommended; not required)

Numerical Optimization, Jorge Nocedal and Stephen Wright, second edition, Springer-Verlag, 2006.

Other material will be passed out as notes.


The course requirements include class attendance, seven or eight homework assignments, and an in-class midterm. Depending on enrollment, there will also be a final exam or an original project.


HW1, due September 16, 2014.
HW2, due September 24, 2014.
HW3, due October 1, 2014.
HW4, due October 8, 2014.
HW5, due November 17, 2014.
HW6, due December 4, 2014.
Homeworks must be submitted in electronic form, and must be emailed to the instructor before midnight on the due date. Without explicit permission from the instructor in advance, late homework will be marked down by 30% for every day of lateness.


Handout 1.


Students are expected to be comfortable with numerical linear algebra and multivariate calculus, and to have programming experience (preferably in Matlab). Students without all elements of this background are likely to have difficulty with the course material.


Most homework assignments will include programming and analysis of the numerical results. Students will be asked to submit their code as part of each homework assignment. The instructor will use Matlab, an interactive software package and programming environment, for her own programs. If you would like to use another language, please obtain advance permission from the instructor. Matlab is a product of the Mathworks; a student version costs around $100 at the Computer Store, or you can use Matlab in a Courant computer lab. (You will need a CIMS account.) You can use Matlab remotely, with a few (solvable) complications if you wish to use its graphics capabilities.


There will be an in-class, closed-book midterm during the second hour of class on October 28, 2014.

Course Project

Each student is expected to choose and complete an individual project to demonstrate his/her creativity and mastery of important concepts of numerical optimization. Course projects are due in electronic form by 11:59pm on December 12, 2012. Details about the project requirement, including deadlines and a list of possible projects, are given at Project information.