Applications of 360 Degree Cameras
This project involves the application of our work on catadioptric cameras with wide fields of view. We have developed compact 360 degree cameras that have been mounted on a variety of robots and used to control or even drive the robots from remote locations. We have also developed intelligent surveillance systems that use 360 degree video to simultaneously track multiple objects moving within the large field of view. A perspective video stream is computed from the omnidirectional video stream that seeks to keep the moving objects within its field of view. In another project, we have combined a 360 degree camera (master) with a conventional pan/tilt/zoom (PTZ) camera (slave) and used the 360 degree camera to determine where the PTZ camera should look next. We have also developed an imaging system called the Zoomnicam which is an omnidirectional camera with a very wide range of optical zoom settings. In this system, the curved mirror is mounted on a controllable translational stage and the optics includes a controllable zoom lens. This enables the system to go from super-wide angle imaging to high-zoom imaging by simply translating the mirror and changing the optical zoom of the lens.

Publications

"Evolution of Urban Robotics System Developments,"
H. Schempf, E. Mutshler, B. Chemel, S.K. Nayar, C. Piepgras, W. Crowley and S. Boehmke,
IEEE/ASME International Conference on Advanced Intelligent Mechatronics,
pp.689-694, Sep, 1999.
[PDF] [bib] [©]

"Omnidirectional Vision Systems: 1998 PI Report,"
S.K. Nayar and T.E. Boult,
DARPA Image Understanding Workshop (IUW),
pp.93-99, Nov, 1998.
[PDF] [bib] [©]

"Remote Reality via Omnidirectional Imaging,"
T.E. Boult,
DARPA Image Understanding Workshop (IUW),
pp.1049-1052, Nov, 1998.
[PDF] [bib] [©]

"Catadioptric Video Sensors,"
S.K. Nayar, J. Gluckman, R. Swaminathan, S. Lok and T.E. Boult,
Workshop on Applications of Computer Vision (WACV),
pp.236-237, Oct, 1998.
[PDF] [bib] [©]

"Ego-motion and Omnidirectional Cameras,"
J.M. Gluckman and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
pp.999-1005, Jan, 1998.
[PDF] [bib] [©]

"Omnidirectional VSAM Systems: PI Report,"
S.K. Nayar and T.E. Boult,
DARPA Image Understanding Workshop (IUW),
pp.55-62, 1997.
[bib] [©]

Videos

If you are having trouble viewing these .avi videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.

  Tactical Mobile Robot:

This robot was developed under a joint project with the Jet Propulsion Laboratory, Carnegie Mellon University and IS Robotics (now iRobot). Mounted on the robot is a 360 degree camera developed by the CAVE laboratory at Columbia. It is a folded design and the detector is sensitive to visible light as well as infra-red. At the end of the clip you will see the raw video from the camera.

     
  Passive Cyclops:

The cyclops was developed by Hagen Schempf's group at Carnegie Mellon University. It is about 6 inches in diameter and has a folded 360 degree (omni)camera, video transmitted, microphones and batteries within it. It is a passive mechanical device that can be rolled into an area. It has been mechanically designed so as to always try and keep the optical axis of the omnicamera vertical. A remote user can use mapping software to view the world around the Cyclops as it rolls.

     
  Active Cyclops:

The second version of Cyclops, also developed by Hagen Schempf's group at Carnegie Mellon University, is an active device. It includes actuators that enables a user to remotely control its rolling. Therefore, if the user is not satisfied with the location of the Cyclops, or would like another view, he/she can simply drive it while watching the immersive video it transmits.

     
  ParaRover:

This is a remote controlled car with a 360 degree camera, microphones, speakers and wireless video/audio transceivers. This vehicle can be used to explore a remote site, such as a nuclear plant or any other environment that is hazardous to humans. A remote user can drive the car as well as communicate with people in the remote area via the audio devices mounted on it.

     
  Remote Driving of the ParaRover:

A person can drive the ParaRover using the Remote Reality system developed by Terry Boult at the University of Colorado, Colorado Springs. By wearing the head-mounted display, the user gets the impression of being located on the rover itself. This enables him/her to maneuver the vehicles with ease.

     
  Omnidirectional + PTZ System:

This video demonstrates an integrated monitoring system that includes a 360 degree camera and a conventional pan-tilt-zoom (PTZ) camera. Here, the 360 degree video provides the observer a global view of the region of interest. When an activity is detected, the 360 degree video is used as an interface to drive the PTZ system which then zooms into the activity region to provide greater detail.

     
  Multi-body Omnidirectional Tracking System:

This video shows an intelligent tracking system that automatically tracks multiple objects within the omnidirectional field of view. The system generates a perspective view for each moving object and remains focused on the object as it moves around.

     
  Zoomnicam:

Wide angle cameras, like the above 360 degree cameras, are inherently limited in resolution due to their large fields of view. The zoomnicam is a novel imaging system that has a very wide optical zoom range that can vary from the narrow field of view of a conventional camera to the hemispherical field of view of an omnidirectional camera. This is done by mounting a curved mirror on a translational stage and using a zoom lens between the curved mirror and the detector. This way any desired section of curved mirror can be imaged. This video shows the principle underlying the zoomnicam. (With audio)

     
  PTZ Emulation using Zoomnicam:

This video shows how the Zoomnicam can be used to emulate a PTZ camera but with small linear displacements of the curved mirror. (With audio)

     

Catadioptric Cameras for 360 Degree Imaging

Non-Single Viewpoint Imaging: Raxels and Caustics

Wide Angle Lenses and Polycameras

The World In An Eye

Catadioptric Stereo: Planar and Curved Mirrors