Conventional vision systems and algorithms assume the camera to have a single viewpoint. However, cameras need not always maintain a single viewpoint. For instance, an incorrectly aligned imaging system could cause non-single viewpoints. Also, systems could be designed on purpose to deviate from a single viewpoint to trade-off image characteristics such as resolution and field of view. In these cases, the locus of viewpoints forms what is called a caustic. In this project, we are interested in the analysis, implementation and calibration on non-single viewpoint imaging systems.

In our initial work in this area, we conducted an in-depth analysis of the caustics of catadioptric cameras with conic reflectors. The properties of caustics with respect to field of view and resolution were analyzed. Next, we developed ways to calibrate conic catadioptric systems and estimate their caustics from known camera motion.

In the second phase of this project, we developed a general imaging model that can be used to represent an arbitrary imaging system, including single viewpoint and non-single viewpoint ones. All imaging systems perform a mapping from incoming scene rays to photo-sensitive elements on the image detector. This mapping can be conveniently described using a set of virtual sensing elements called raxels. Raxels include geometric, radiometric and optical properties. We have developed a simple calibration method that uses structured light patterns to extract the raxel parameters of an arbitrary imaging system. Calibration experiments have been conducted for perspective as well as non-perspective imaging systems.

Publications

"The Raxel Imaging Model and Ray-Based Calibration,"
M.D. Grossberg and S.K. Nayar,
International Journal on Computer Vision,
Vol.61, No.2, pp.119-137, Feb, 2005.
[PDF] [bib] [©]

"Designing Mirrors for Catadioptric Systems that Minimize Image Errors,"
R. Swaminathan, S.K. Nayar, and M.D. Grossberg,
Fifth Workshop on Omnidirectional Vision,
May, 2004.
[PDF] [bib] [©]

"A Perspective on Distortions,"
R. Swaminathan, M.D. Grossberg and S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol.II, pp.594-601, Jun, 2003.
[PDF] [bib] [©]

"Caustics of Catadioptric Cameras,"
R. Swaminathan, M.D. Grossberg and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
Vol.2, pp.2-9, Jul, 2001.
[PDF] [bib] [©]

"A General Imaging Model and a Method for Finding its Parameters,"
M.D. Grossberg and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
Vol.2, pp.108-115, Jul, 2001.
[PDF] [bib] [©]

"Non-Metric Calibration of Wide-Angle Lenses and Polycameras,"
R. Swaminathan and S.K. Nayar,
IEEE Transactions on Pattern Analysis and Machine Intelligence,
Vol.22, No.10, pp.1172 - 1178, Oct, 2000.
[PDF] [bib] [©]

Images

		Non-Single Viewpoint Imaging System: This picture shows a few examples of well-known non-single viewpoint imaging systems. They are catadioptric cameras with complex mirrors, wide-angle cameras with fish-eye lenses, camera clusters that capture wide fields of view and cameras (or eyes) that use compound lenses.
		General Imaging Model: In general, an imaging system can be be viewed as performing a specific mapping from scene rays to image pixels. For non-single viewpoint systems, this mapping can be complex.
		Raxels: A general imaging system can be viewed as a collection of raxels, or ray-pixels, where each raxel is a photosensitive element with accompanying optics that allow it to measure essentially a single ray of light in space. In addition to its location and orientation, a raxel may have complex radiometric and optical properties.
		Caustics: The rays captured by the raxels of a general imaging model can be viewed as tangents to a surface called a caustic. The caustic can be conveniently used to model the geometric properties of an imaging system.
		Ray Based Calibration: This picture shows the ray-based calibration methods used to compute the raxels of a general imaging system. The imaging system shows structured light patterns on two planes. The correspondence between pixels in the imaging system and points on the two planes determines the ray-to-pixel mapping for the imaging system.
		Computed Caustic: This picture shows the caustic computed for the perspective (single viewpoint) video camera. As expected. it is a compact cluster.
		Computed Caustic: This picture shows the caustic computed for the non-single viewpoint catadioptric camera. As expected, the caustic is a large surface in this case.

Videos

		Calibration of a Perspective Camera: This video shows the calibration being done for the simplest case of a perspective camera.
		Camera Output: This video shows the raw video produced by the perspective video camera during the calibration procedure.
		Calibration of a Catadioptric Camera: This video shows the calibration being done for a non-single viewpoint catadioptric camera that consists of a perspective lens and a parabolic mirror.
		Camera Output: This video shows the raw video produced by the catadioptric camera during the calibration procedure.

Slides

ICCV 2001 presentation (General Imaging Model)     With videos (zip file)

ICCV 2001 presentation (Caustics)

Software

REFLEX: Reflectors for Flexible Imaging and Projection

Related Projects

Catadioptric Cameras for 360 Degree Imaging

Applications of 360 Degree Cameras

The World In An Eye

Spherical Mosaics: Regular and Stereoscopic

Catadioptric Stereo: Planar and Curved Mirrors

Wide Angle Lenses and Polycameras

Radiometric Camera Calibration