Applications of 360 Degree Cameras |
 | | This project involves the application of our work on catadioptric cameras with
wide fields of view. We have developed compact 360 degree cameras that have
been mounted on a variety of robots and used to control or even drive the
robots from remote locations. We have also developed intelligent surveillance
systems that use 360 degree video to simultaneously track multiple objects
moving within the large field of view. A perspective video stream is computed
from the omnidirectional video stream that seeks to keep the moving objects
within its field of view. In another project, we have combined a 360 degree
camera (master) with a conventional pan/tilt/zoom (PTZ) camera (slave) and used
the 360 degree camera to determine where the PTZ camera should look next. We
have also developed an imaging system called the Zoomnicam which is an
omnidirectional camera with a very wide range of optical zoom settings. In this
system, the curved mirror is mounted on a controllable translational stage and
the optics includes a controllable zoom lens. This enables the system to go
from super-wide angle imaging to high-zoom imaging by simply translating the
mirror and changing the optical zoom of the lens. |
Publications
"Evolution of Urban Robotics System Developments," H. Schempf, E. Mutshler, B. Chemel, S.K. Nayar, C. Piepgras, W. Crowley and S. Boehmke, IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp.689-694, Sep, 1999. [PDF] [bib] [©]
"Omnidirectional Vision Systems: 1998 PI Report," S.K. Nayar and T.E. Boult, DARPA Image Understanding Workshop (IUW), pp.93-99, Nov, 1998. [PDF] [bib] [©]
"Remote Reality via Omnidirectional Imaging," T.E. Boult, DARPA Image Understanding Workshop (IUW), pp.1049-1052, Nov, 1998. [PDF] [bib] [©]
"Catadioptric Video Sensors," S.K. Nayar, J. Gluckman, R. Swaminathan, S. Lok and T.E. Boult, Workshop on Applications of Computer Vision (WACV), pp.236-237, Oct, 1998. [PDF] [bib] [©]
"Ego-motion and Omnidirectional Cameras," J.M. Gluckman and S.K. Nayar, IEEE International Conference on Computer Vision (ICCV), pp.999-1005, Jan, 1998. [PDF] [bib] [©]
"Omnidirectional VSAM Systems: PI Report," S.K. Nayar and T.E. Boult, DARPA Image Understanding Workshop (IUW), pp.55-62, 1997. [bib] [©]
|
Videos
If you are having trouble viewing these .avi videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.
 |
|
Tactical Mobile Robot:
This robot was developed under a joint project with the Jet Propulsion
Laboratory, Carnegie Mellon University and IS Robotics (now iRobot). Mounted on
the robot is a 360 degree camera developed by the CAVE laboratory at Columbia.
It is a folded design and the detector is sensitive to visible light as well as
infra-red. At the end of the clip you will see the raw video from the
camera.
|
| |
|
|
 |
|
Passive Cyclops:
The cyclops was developed by Hagen
Schempf's group at Carnegie Mellon University. It is about 6 inches in
diameter and has a folded 360 degree (omni)camera, video transmitted,
microphones and batteries within it. It is a passive mechanical device that can
be rolled into an area. It has been mechanically designed so as to always try
and keep the optical axis of the omnicamera vertical. A remote user can use
mapping software to view the world around the Cyclops as it rolls.
|
| |
|
|
 |
|
Active Cyclops:
The second version of Cyclops, also developed by Hagen
Schempf's group at Carnegie Mellon University, is an active device. It
includes actuators that enables a user to remotely control its rolling.
Therefore, if the user is not satisfied with the location of the Cyclops, or
would like another view, he/she can simply drive it while watching the
immersive video it transmits.
|
| |
|
|
 |
|
ParaRover:
This is a remote controlled car with a 360 degree camera, microphones,
speakers and wireless video/audio transceivers. This vehicle can be used to
explore a remote site, such as a nuclear plant or any other environment that is
hazardous to humans. A remote user can drive the car as well as communicate
with people in the remote area via the audio devices mounted on it.
|
| |
|
|
 |
|
Remote Driving of the ParaRover:
A person can drive the ParaRover using the Remote Reality system developed by
Terry Boult at the University of Colorado, Colorado Springs. By wearing the
head-mounted display, the user gets the impression of being located on the
rover itself. This enables him/her to maneuver the vehicles with ease.
|
| |
|
|
 |
|
Omnidirectional + PTZ System:
This video demonstrates an integrated monitoring system that includes a 360 degree camera and a conventional pan-tilt-zoom (PTZ) camera. Here, the 360 degree video provides the observer a global view of the region of interest. When an activity is detected, the 360 degree video is used as an interface to drive the PTZ system which then zooms into the activity region to provide greater detail.
|
| |
|
|
 |
|
Multi-body Omnidirectional Tracking System:
This video shows an intelligent tracking system that automatically tracks
multiple objects within the omnidirectional field of view. The system generates
a perspective view for each moving object and remains focused on the object as
it moves around.
|
| |
|
|
 |
|
Zoomnicam:
Wide angle cameras, like the above 360 degree cameras, are inherently limited
in resolution due to their large fields of view. The zoomnicam is a novel
imaging system that has a very wide optical zoom range that can vary from the
narrow field of view of a conventional camera to the hemispherical field of
view of an omnidirectional camera. This is done by mounting a curved mirror on
a translational stage and using a zoom lens between the curved mirror and the
detector. This way any desired section of curved mirror can be imaged. This
video shows the principle underlying the zoomnicam. (With audio)
|
| |
|
|
 |
|
PTZ Emulation using Zoomnicam:
This video shows how the Zoomnicam can be used to emulate a PTZ camera but
with small linear displacements of the curved mirror. (With audio)
|
| |
|
|
|
Catadioptric Cameras for 360 Degree Imaging
Non-Single Viewpoint Imaging: Raxels and Caustics
Wide Angle Lenses and Polycameras
The World In An Eye
Catadioptric Stereo: Planar and Curved Mirrors
|
|
|