Flexible Depth of Field Photography

The range of scene depths that appear focused in an image is known as the depth of field (DOF). Conventional cameras are limited by a fundamental trade-off between DOF and signal-to-noise ratio (SNR). For a dark scene, the aperture of the lens must be opened up to maintain SNR, which causes the DOF to reduce. Also, today’s cameras have DOFs that correspond to a single slab that is perpendicular to the optical axis. In this project, we developed an imaging system that enables one to manipulate the DOF in new and powerful ways. We propose to vary the position and/or orientation of the image detector, during the integration time of a single photograph. Even when the detector motion is very small (tens of microns), a large range of scene depths (several meters) is captured both in and out of focus.

We demonstrate four applications of flexible DOF. First, extended DOF, where a large depth range is captured with a very wide aperture (low noise) but with nearly depth-independent defocus blur. Applying deconvolution to a captured image gives an image with extended DOF and yet high SNR. We also show that we can capture scenes with discontinuous DOFs. For instance, near and far objects can be imaged with sharpness while objects in between are severely blurred. We show that our camera can also capture images with tilted DOFs (Scheimpflug imaging) without tilting the image detector. Finally, we show how our camera can capture scenes with non-planar DOFs.

Publications

"Flexible Depth of Field Photography,"
H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar,
European Conference on Computer Vision (ECCV),
Oct, 2008.
[PDF] [bib] [©]

Images

  Principle:

We propose to translate the detector along the optical axis during image integration. Consequently, while the detector is collecting photons for a photograph, a large range of scene depths come into and go out of focus. We demonstrate that by controlling how we translate the detector, we can manipulate the depth of field of the imaging system.

     
  Prototype System:

In our prototype, the detector is mounted on a translation stage that is controlled by a micro-actuator.

     
  Detector Motion for Extended DOF:

To capture a scene with a extended depth of field, while using a large aperture for good SNR, we propose to translate a detector with a global shutter at a constant speed during image integration. In a captured image, the blur kernel is almost the same for all scene depths and image locations. Applying deconvolution with a single blur kernel gives a sharp, all in focus image.

     
  Extended DOF - Chess:

Example that demonstrates that our camera can capture scenes with a large depth of field while using a large aperture to ensure high SNR.

     
  Extended DOF - Three Faces:

Example that demonstrates that our camera can capture scenes with a large depth of field while using a large aperture to ensure high SNR.

     
  Extended DOF - Soft Toys:

Example that demonstrates that our camera can capture scenes with a large depth of field while using a large aperture to ensure high SNR.

     
  Extended DOF - Cathedral at Night (Low Light Imaging):

In a normal camera, the tradeoff between depth of field and SNR is extreme for dimly lit scenes such as this one. On the other hand, our camera is able to capture such a scene with a large depth of field and good SNR.

     
  Detector Motion for Discontinuous DOF:

To capture a scene with a discontinuous depth of field, we propose to translate a detetector with a global shutter in a non-linear fashion during image integration. The detector motion shown in the animation enables us to capture two disconnected scene regions with sharpness while objects in between are severly blurred.

     
  Discontinuous DOF - Vanishing Wire Mesh:

In this scene we have a toy cow and a toy hen in front of a wire mesh, behind which there is a scenic background. We can capture this scene so that we see the toys and the background, but the wire mesh is so defocused that it vanishes from the image.

     
  Detector Motion for Tilted DOF:

To capture a scene with a tilted depth of field, we propose to translate a detector with a rolling shutter at a constant speed during image integration. This emulates a tilted image detector, and as a result we get a tilted depth of field. By varying the translation speed, we can control the tilt.

     
  Tilted DOF - Coffee Mug on Newspaper:

In this example, the table-top with the newspaper, mug, and keys is tilted with the lens plane. We set the detector translation in order to realize a tilted depth of field that is aligned with the table top. The entire newspaper is now in focus. The bottom of the mug is focused, but the top is not, indicating that the depth of field is aligned with the table top.

     
  Detector Motion for Non-Planar DOF:

To capture a scene with a non-planar depth of field, we propose to translate a detector with a rolling shutter in a non-linear fashion. This emulates a curved image detector, and as a result we get a curved/non-planar depth of field.

     
  Non-Planar DOF - Crayons for Sale:

In this example, the crayons are arranged on a semi-circular arc, while the price tag is placed at the same depth as the nearest crayons. We capture this scene with a curved (non-planar) depth of field that is aligned with the crayons, so that all the crayons are in focus, while the price tag is defocused.

     
  Extended DOF using SLR's Focus Ring:

We can exploit the camera's focusing mechanism to manipulate depth of field. To capture a scene with an extended depth of field, we propose to uniformly rotate the focus ring of a SLR camera lens during image integration. This corresponds to moving the lens along the optical axis at roughly constant speed.

     
  Extended DOF using SLR's Focus Ring - Flowerbed:

In this example, we demonstrate that by uniformly rotating a SLR camera len's focus ring during image integration, we can capture scenes with a large depth of field while using a large aperture to ensure high SNR.

     
  Extended DOF using SLR's Focus Ring - Lawn:

In this example, we demonstrate that by uniformly rotating a SLR camera len's focus ring during image integration, we can capture scenes with a large depth of field while using a large aperture to ensure high SNR.

     
  Computing an All-In-Focus Image from a Focal Stack:

In this example, we demonstrate how we can compute an all-in-focus image from a focal stack using our approach for extended depth of field imaging.

     
  Detector Motion for Extended DOF Video:

To capture extended depth of field video, we propose to move the detector at a constant speed, forward one frame, back the next and so on. This is possible because the blur kernel is invariant to the direction of motion.

     

Video

If you are having trouble viewing these videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.

  Extended DOF Video:

Video sequences comparing the depths of field obtained from our camera and from a normal camera.

     

Motion-Invariant Photography (MIT)

Super-Resolution: Jitter Camera

Depth from Defocus