Stereopsis


(a)     (b)     (a)

(c)     (d)

Figures (a) and (b) shows a stereo pair, left and right images, of the pentagon scene. Figure (a) is repeated for cross fusers. It is 8-bits and of size 512 x 512 pixels. Epipolar lines (see figure (e)) are assumed to be horizontal lines. Figure (c) shows the disparity map (correspondence map), where the grey value (1 to 15) correspond to the disparity (x-displacement) between corresponding pair of pixels. Figure (d) shows the pixels that are half-occluded.

What is Stereo ?

Binocular stereo is the process of obtaining depth information from a pair of cameras (see Figure (a) and (b)). Typically, depth is directly related to the disparity map (see Figure (c)). Stereo produces a grouping mechanism to resolve the optimal surface (the disparity map) given the stereo pair of images.

(e)    


Epipolar line: A point in the right frame and the focal point defines a 3-D line that projects to the other frame (via a focal point) to give the epipolar line. Clear is that a given point in one frame and the two focal points define a plane that crosses the two image planes to yield corresponding epipolar lines.


Research

Our research focus on the view that the stereo problem is of reconstructing the depth surface, in particular issues of half-occlusions and discontinuities of surfaces, from a Bayesian view. We are also motivated by cognitive problems of understanding illusory surfaces and transparency. Our objective is to produce algorithms that are simple (small size of code and parallelizable), accuracy (subpixel) and robust (work extremely well on various conditions). Lately, we have been interested on algorithms and models that address the epipolar lines (see figure (e)) interactions and on combining stereo and shape from shading.

PAPERS and ABSTRACTS

Occlusions, Discontinuities, and Epipolar Lines in Stereo

H. Ishikawa and D. Geiger

Fifth European Conference in Computer Vision (ECCV'98), 1998.

Binocular stereo is the process of obtaining depth information from a pair of left and right views of a scene. We present a new approach to compute the disparity map by solving a global optimization problem that models occlusions, discontinuities, and epipolar-line interactions. In the model, geometric constraints require every disparity discontinuity along the epipolar line in one eye to {\it always} correspond to an occluded region in the other eye, while at the same time encouraging smoothness across epipolar lines. Smoothing coefficients are adjusted according to the edge and junction information. For some well-defined set of optimization functions, we can map the optimization problem to a maximum-flow problem on a directed graph in a novel way, which enables us to obtain a global solution in a polynomial time. Experiments confirm the validity of this approach.

Occlusions and Binocular Stereo

D. Geiger and B. Ladendorf and A. Yuille

International Journal of Computer Vision (IJCV), Vol. 14, pp-211--226, 1995.

Binocular stereo is the process of obtaining depth information from a pair of cameras. In the past, stereo algorithms have had problems at occlusions and have tended to fail there (though sometimes post-processing has been added to mitigate the worst effects). We show that, on the contrary, occlusions can help stereo computation by providing cues for depth discontinuities. We describe a theory for stereo based on the Bayesian approach, using adaptive windows and a prior weak smoothness constraint, which incorporates occlusion. Our model assumes that a disparity discontinuity, along the epipolar line, in one eye {\it always} corresponds to an occluded region in the other eye thus, leading to an {\it occlusion constraint}. This constraint restricts the space of possible disparity values, thereby simplifying the computations. An estimation of the disparity at occluded features is also discussed in light of psychophysical experiments. Using dynamic programming we can find the optimal solution to our system and the experimental results are good and support the assumptions made by the model.

Stereo Vision via Markov Models

D. Geiger and L. Maloney and K. Kumaran

Allerton Conference in Computer Science, 1995.

Binocular stereo is the process of obtaining depth information from two simultaneous views of a scene. The performance of computational stereo algorithms remains distinctly inferior to human performance, and at the same time, many phenomena in human stereo vision (transparency, Da Vinci stereopsis, subjective contours) have no natural place in these algorithms.

Through the framework of hidden Markov models we formulate a stereo algorithm that match along epipolar lines (one dimensional). Our aim is to introduce, in a simple and yet general manner, constraints that can account for these psychophysical phenomena. The first result is a distinctive neighborhood structure for our model and these various properties of stereo, such as occlusions, disparity discontinuity, transparency, are accounted. In order for us to fully address the problem of subjective contours the model is extended to two dimensions and a Markov random field model emerges. Finally we discuss the primitives of stereo, proposing differential geometric features, and studying their relation to occlusions, discontinuities and transparency. Initial experiments demonstrate the power of our formulation.

Here are papers that I do not have online

Stereopsis and eye movement,
D. Geiger and A. Yuille (1989)
Biol. Cybern 62 (2): 117-128.

Stereo, mean field theory and psychophysics,
A. Yuille, D.Geiger and H. H. Bulthoff (1991)
Network: Computation in Neural Systems. Vol 2, pp 423-442.