Conventional stereo uses two or more cameras to compute three-dimensional scene structure. Catadioptric stereo enables the capture of multiple views of a scene using a single camera. In this project, we are exploring the use of planar as well as curved mirrors to develop catadioptric stereo systems.

By placing planar mirrors in front of the camera, multiple virtual cameras are created. We have studied the geometric properties and self-calibration of planar catadioptric systems in detail. A couple of prototypes have been developed. In addition, a real-time stereo algorithm has been developed that computes depth maps at frame rate. The use of a single video camera ensures that the radiometric properties of both views of the scene are identical. This leads to more robust correspondence and hence depth estimation.

Traditional stereo systems have small fields of view which limits their usefulness in some applications. We have developed a compact panoramic stereo camera that includes two panoramic sensors with aligned optical axes. Each panoramic sensor uses a parabolic mirror and a telecentric lens. A stereo algorithm has been developed that first maps the parabolic images to cylindrical panoramas. As a result, epipolar lines in both images are vertical scan lines. This naturally increases the efficiency of the stereo search algorithm. A stereo algorithm has been developed that produces panoramic depth video at frame-rate. We have also analyzed the epipolar geometries of a complete class of catadioptric stereo cameras that use mirrors with conic sections (hyperbolic, parabolic, ellipsoidal, conical).

Publications

"Rectified Catadioptric Stereo Sensors,"
J.M. Gluckman and S.K. Nayar,
IEEE Transactions on Pattern Analysis and Machine Intelligence,
Vol.24, No.2, pp.224-236, Feb, 2002.
[PDF] [bib] [©]

"Rectifying Transformations that Minimize Resampling Effects,"
J.M. Gluckman and S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol.I, pp.111-117, Dec, 2001.
[PDF] [bib] [©]

"Catadioptric Stereo Using Planar Mirrors,"
J. Gluckman and S.K. Nayar,
International Journal on Computer Vision,
Vol.44, No.1, pp.65-79, Aug, 2001.
[PDF] [bib] [©]

"Rectified Catadioptric Stereo Sensors,"
J. Gluckman and S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol.2, pp.380-387, Jun, 2000.
[PDF] [bib] [©]

"Planar Catadioptric Stereo: Geometry and Calibration,"
J. Gluckman and S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol.1, pp.22-28, Jun, 1999.
[PDF] [bib] [©]

"A Real-Time Catadioptric Stereo System using Planar Mirrors,"
J.M. Gluckman and S.K. Nayar,
DARPA Image Understanding Workshop (IUW),
pp.309-313, Nov, 1998.
[PDF] [bib] [©]

"Real-Time Omnidirectional and Panoramic Stereo,"
J.M. Gluckman, S.K. Nayar and K.J. Thoresz,
DARPA Image Understanding Workshop (IUW),
pp.299-303, Nov, 1998.
[PDF] [bib] [©]

"Ego-motion and Omnidirectional Cameras,"
J.M. Gluckman and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
pp.999-1005, Jan, 1998.
[PDF] [bib] [©]

"Stereo with Mirrors,"
S. Nene and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
pp.1087-1094, Jan, 1998.
[PDF] [bib] [©]

"Sphereo: Determining Depth using Two Specular Spheres and a Single Camera,"
S.K. Nayar,
SPIE Conference on Optics, Illumination, and Image Sensing for Machine Vision III,
pp.245-254, Nov, 1988.
[PDF] [bib] [©]

Images

		Catadioptric Stereo Camera with Planar Mirrors: This image shows the stereo camera we have developed that uses two mirrors and a single detector.
		Rectified Catadioptric Stereo Camera: This lens attachment allows one to convert a digital still or video camera into a stereo camera that produces two rectified views.
		Bench Scene: This picture shows the single split-shot image of a scene captured with the above camera and a stereo image computed from it. The stereo image must be viewed with red-blue glasses to perceive scene depth.
		Stairway Scene: This picture shows another split-shot image of a scene captured with the above camera and the stereo image computed from it. The stereo image must be viewed with red-blue glasses to perceive scene depth.
		Panoramic Stereo Camera: This system uses two aligned catadioptric video cameras, each with a 360 degree field of view.
		Stereo Panoramic Images: This picture shows the raw output (two panoramic images) of the above panoramic stereo camera.
		Panoramic Depth Maps: This picture shows a panoramic stereo pair computed from the above raw stereo pair, and the depth (top image) of the scene computed from the panoramic stereo pair.

Video

		Catadioptric Stereo Video: This video shows the raw video stream produced by the stereo camera that uses tilted planar mirrors (left) and the depth video computed from it in real time (right).

Related Projects

Catadioptric Cameras for 360 Degree Imaging

Applications of 360 Degree Cameras

The World In An Eye

Spherical Mosaics: Regular and Stereoscopic

Specularities in Stereo and Motion