Slides and Notes for SIGGRAPH Course 30:Performance-Driven Facial Animation Section: Markerless Face Capture and Automatic Model Construction Chris Bregler  chris.bregler@nyu.edu Latest Version: http://cs.nyu.edu/~bregler/sig-course-06-face/

Slides click here: bregler-slides.ppt

In a way, "Makerless Face Capture" for animation is as old as the trade of traditional animation, taught by Disney and others in the early 20th century. Animators always look at example recordings, especially for facial animations. For example, the idiosyncrasies and mannerisms of the Step-Mother's facial expressions in Cinderella were based on filmed recordings of actress Eleanor Audley. In some cases, the animations were even a direct copy of the hand-traced motions from film, like in scenes of Snow White. Performing such tracing and tracking automatically by a computer has been the topic of computer vision research over the past 20 years. Just recently new techniques have become available that have good enough quality for animation in the entertainment industry.

In this part of the course we will survey these visual tracking techniques and modeling techniques, with a focus on non-rigid or deformable tracking and models. This includes a review of methods that incorporate non-rigid model constraints into optical-flow based methods and appearance/image based techniques, and blend-shape models or PCA models that are trained from example data. In addition, we will focus on a new family of techniques that evolved over the past five years, so-called “rank-constrained” techniques. These techniques combine the tracking and the 3D non-rigid reconstruction problem into one global XYT optimization problem. We will review applications of all these methods to lip-shape, eye-gaze, head-motion, and full-face tracking and modeling.

Snakes / Active Contours:

Factorization Based Face Capture and Modelling:

Vision Based Face Animation (so far incomplete!):