\\

http://cs.nyu.edu/~fergus/deblur-tn.png |

http://cs.nyu.edu/~fergus/CODED.jpg |

http://cs.nyu.edu/~fergus/dark2.jpg |

My work in computational photography addresses situations where the recorded image is corrupted by blur or noise, for example due to low levels of ambient lighting. Using statistical priors on the image structure, possibly in conjunction with simple modifications to the camera hardware, the corruption can be reduced or removed.

Intro text here.

Joint work with Dilip Krishnan.

Joint work with: B. Singh, A. Hertzmann, S. T. Roweis and W. T. Freeman.

Joint work with: A. Levin, F. Durand and W. T. Freeman.

Dark Flash Photography

Image deblurring

Acquiring depth and an image using a conventional camera with a coded aperture

Dark Flash Photography Project Website

Camera flashes produce intrusive bursts of light that disturb or dazzle. We present a prototype camera and flash that uses infra-red and ultra-violet light mostly outside the visible range to capture pictures in low-light conditions. This “dark” flash is at least two orders of magnitude dimmer than conventional flashes for a comparable exposure. Building on ideas from flash/no-flash photography, we capture a pair of images, one using the dark flash, other using the dim ambient illumination alone. We then exploit the correlations between images recorded at different wavelengths to denoise the ambient image and restore fine details to give a high quality result, even in very weak illumination. The processing techniques can also be used to denoise images captured with conventional cameras.

Image deblurring. Project Webpage

Many photos are spoiled by the user's hand moving while the camera shutter is open. Points in the scene are smeared out over the exposure interval, resulting in a blurry photo. My co-authors and I pose the problem as a blind deconvolution: we assume the blur function is constant over the image and so aim to recover the blur kernel (the motion of the user's hand) together with the underlying sharp image. We make use of heavy-tailed image priors on the image gradients in conjunction with some sophisticated machine learning tools to solve the blind deconvolution problem. We apply the algorithm to real photos, obtaining what we believe are the first convincing results on this difficult problem.

Acquiring depth and an image using a conventional camera with a coded aperture. Project Webpage

Together with Anat Levin, we present a technique that permits BOTH a high resolution image AND depth information to be recovered from a single shot. To do this we make a simple modification to a conventional lens: we insert into the aperture a pattern cut from a piece of cardboard. This pattern changes the out-of-focus blur patterns, facilitating the acquisition of depth information. The local defocus patterns can be analyzed to recover a depth map. Then, using the depth map a high resolution all-focus image can also be recovered.

Computational Photography

Dark Flash Photography

Image deblurring.

Many photos are spoiled by the user's hand moving while the camera shutter is open. Points in the scene are smeared out over the exposure interval, resulting in a blurry photo. My co-authors and I pose the problem as a blind deconvolution: we assume the blur function is constant over the image and so aim to recover the blur kernel (the motion of the user's hand) together with the underlying sharp image. We make use of heavy-tailed image priors on the image gradients in conjunction with some sophisticated machine learning tools to solve the blind deconvolution problem. We apply the algorithm to real photos, obtaining what we believe are the first convincing results on this difficult problem. Project Webpage

Acquiring depth and an image using a conventional camera with a coded aperture.

Together with Anat Levin, we present a technique that permits BOTH a high resolution image AND depth information to be recovered from a single shot. To do this we make a simple modification to a conventional lens: we insert into the aperture a pattern cut from a piece of cardboard. This pattern changes the out-of-focus blur patterns, facilitating the acquisition of depth information. The local defocus patterns can be analyzed to recover a depth map. Then, using the depth map a high resolution all-focus image can also be recovered. Project Webpage