Flexible Depth of Field Photography |
 | |
The range of scene depths that appear focused in an image is
known as the depth of field (DOF). Conventional cameras are limited by
a fundamental trade-off between DOF and signal-to-noise ratio
(SNR). For a dark scene, the aperture of the lens must be opened up to
maintain SNR, which causes the DOF to reduce. Also, today’s cameras
have DOFs that correspond to a single slab that is perpendicular to the
optical axis. In this project, we developed an imaging system that enables
one to manipulate the DOF in new and powerful ways. We propose to
vary the position and/or orientation of the image detector, during the
integration time of a single photograph. Even when the detector motion
is very small (tens of microns), a large range of scene depths (several
meters) is captured both in and out of focus.
We
demonstrate four applications of flexible DOF. First, extended DOF,
where a large depth range is captured with a very wide
aperture (low noise) but with nearly depth-independent defocus blur.
Applying deconvolution to a captured image gives an image with extended DOF and yet high SNR. We also show that we can capture scenes
with discontinuous DOFs. For instance, near and far objects can be imaged with sharpness
while objects in between are severely blurred.
We show that our camera can also capture images with tilted DOFs
(Scheimpflug imaging) without tilting the image detector.
Finally, we show how our camera can capture scenes with non-planar DOFs.
|
Publications
"Flexible Depth of Field Photography," H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, European Conference on Computer Vision (ECCV), Oct, 2008. [PDF] [bib] [©]
|
Images
 |
|
Principle:
We propose to translate the detector along the optical axis during image integration.
Consequently, while the detector is collecting photons for a photograph, a large range of scene depths come into and go out of focus. We demonstrate that by controlling how we translate the detector, we can manipulate the depth of field of the imaging system.
|
| |
|
|
 |
|
Prototype System:
In our prototype, the detector is mounted on a translation stage that is controlled
by a micro-actuator.
|
| |
|
|
 |
|
Detector Motion for Extended DOF:
To capture a scene with a extended depth of field,
while using a large aperture for good SNR, we propose
to translate a detector with a global shutter at a constant speed during image integration.
In a captured image, the blur kernel is almost the same for all scene depths
and image locations. Applying deconvolution with a single blur kernel
gives a sharp, all in focus image.
|
| |
|
|
 |
|
Extended DOF - Chess:
Example that demonstrates that our camera can capture
scenes with a large depth of field while using a large aperture to ensure high SNR.
|
| |
|
|
 |
|
Extended DOF - Three Faces:
Example that demonstrates that our camera can capture
scenes with a large depth of field while using a large aperture to ensure high SNR.
|
| |
|
|
 |
|
Extended DOF - Soft Toys:
Example that demonstrates that our camera can capture
scenes with a large depth of field while using a large aperture to ensure high SNR.
|
| |
|
|
 |
|
Extended DOF - Cathedral at Night (Low Light Imaging):
In a normal camera, the tradeoff between depth of field
and SNR is extreme for dimly lit scenes such as this one.
On the other hand, our camera is able to capture such a scene with a large depth of field and good SNR.
|
| |
|
|
 |
|
Detector Motion for Discontinuous DOF:
To capture a scene with a discontinuous depth of field, we propose to
translate a detetector with a global shutter in a non-linear fashion
during image integration. The
detector motion shown in the animation enables us to capture two disconnected scene regions with
sharpness while objects in between are severly blurred.
|
| |
|
|
 |
|
Discontinuous DOF - Vanishing Wire Mesh:
In this scene we have a toy cow and a toy hen
in front of a wire mesh, behind which there is a scenic background.
We can capture this scene so that we see the toys and the background,
but the wire mesh is so defocused that it vanishes from the image.
|
| |
|
|
 |
|
Detector Motion for Tilted DOF:
To capture a scene with a tilted depth of field,
we propose to translate a detector with a rolling shutter
at a constant speed during image integration.
This emulates a tilted image detector, and as a result
we get a tilted depth of field. By varying
the translation speed, we can control the tilt.
|
| |
|
|
 |
|
Tilted DOF - Coffee Mug on Newspaper:
In this example, the table-top with the newspaper, mug, and
keys is tilted with the lens plane. We set the detector
translation in order to realize a tilted depth of field
that is aligned with the table top. The entire newspaper
is now in focus. The bottom of the mug is focused,
but the top is not, indicating that the depth of field is aligned
with the table top.
|
| |
|
|
 |
|
Detector Motion for Non-Planar DOF:
To capture a scene with a non-planar depth of field,
we propose to translate a detector with a rolling shutter
in a non-linear fashion. This emulates a curved image
detector, and as a result we get a curved/non-planar depth of field.
|
| |
|
|
 |
|
Non-Planar DOF - Crayons for Sale:
In this example, the crayons are arranged on a semi-circular arc, while the
price tag is placed at the same depth as the nearest crayons. We
capture this scene with a curved (non-planar) depth of field that
is aligned with the crayons,
so that all the crayons are in focus, while the price tag is defocused.
|
| |
|
|
 |
|
Extended DOF using SLR's Focus Ring:
We can exploit the camera's focusing mechanism to manipulate depth of field. To
capture a scene with an extended depth of field, we propose to uniformly rotate the focus
ring of a SLR camera lens during image integration. This corresponds to moving
the lens along the optical axis at roughly constant speed.
|
| |
|
|
 |
|
Extended DOF using SLR's Focus Ring - Flowerbed:
In this example, we demonstrate that by uniformly rotating a SLR camera len's focus ring during image integration,
we can capture scenes with a large depth of field while using a large aperture to ensure high SNR.
|
| |
|
|
 |
|
Extended DOF using SLR's Focus Ring - Lawn:
In this example, we demonstrate that by uniformly rotating a SLR camera len's focus ring during image integration,
we can capture scenes with a large depth of field while using a large aperture to ensure high SNR.
|
| |
|
|
 |
|
Computing an All-In-Focus Image from a Focal Stack:
In this example, we demonstrate how we can compute an all-in-focus
image from a focal stack using our approach for extended depth of
field imaging.
|
| |
|
|
 |
|
Detector Motion for Extended DOF Video:
To capture extended depth of field video, we propose to move the
detector at a constant speed, forward one frame, back the next and
so on. This is possible because the blur kernel is invariant to
the direction of motion.
|
| |
|
|
|
Video
If you are having trouble viewing these videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.
 |
|
Extended DOF Video:
Video sequences comparing the depths of field obtained from our camera and from a normal camera.
|
| |
|
|
|
Motion-Invariant Photography (MIT)
Super-Resolution: Jitter Camera
Depth from Defocus
|
|
|