We have developed a camera that enables a user to capture (with the press of a button) a stack of images of a scene that correspond to different focus settings. This focal sweep camera (Fosera, for short) uses a high-speed image sensor that is translated with respect to the lens while images are recorded. The captured stack of images is referred to as a duration focal stack - images of a possibly dynamic scene captured while the plane of focus is swept through it. We have derived a condition that determines, for any given imaging system, the sensor speed and shortest duration needed to sweep the depth of field of the imaging system through a desired depth range. In our experiments we have used durations between 200 msec and 500 msec to capture stacks that have between 24 and 60 images. While we have used sensor actuation in our current system, focal sweep can be implemented using other means, such as, the use of a liquid lens whose focal length is varied as a function of time.

While the focal stack produced by a light field camera (such as Lytro) corresponds to a single instant of time, the focal stack captured by Fosera corresponds to a finite duration of time and hence includes scene motion. Put a different way, while the Lytro trades off spatial resolution to capture an instantaneous focal stack, Fosera trades off temporal resolution to maintain high spatial resolution. We have explored the information embedded within a duration focal stack and developed an algorithm for computing a depth map which, in the case of a dynamic scene, includes depth values that correspond to different instants of time. This depth map is used as an index map to facilitate post-focusing over space and time - we refer to this as space-time refocusing.

To show our refocusing results we have developed a click-to-refocus viewer for duration focal stacks. While our current viewer uses a single index map for refocusing, we have also developed a method that tracks objects in the focal stack to compute a different index map for each image in the focal stack. In this setting, clicking on a moving object takes the user to the image in which that object comes into focus.

This work was supported in parts by ONR (Grant Nos. N00014-11-1-0285 and N00014-08-1-0929) and DARPA (Grant No. W911NF-10-1-0214).

Publications

"Focal Sweep Camera for Space-Time Refocusing,"
C. Zhou, D. Miau, and S.K. Nayar,
Technical Report, Department of Computer Science, Columbia University,
Nov, 2012.
[PDF] [bib] [©]

Gallery

		Gallery of "Breathing Pictures": This gallery includes several interactive photos captured using a focal sweep camera. We refer to these as breathing pictures as the user gets a feel for both depth and motion while interacting with a scene.

Video

		Focal Sweep Camera: This video (with audio) shows Fosera in action. Each short buzz in the video corresponds to the capture of a single duration focal stack. Since focal sweep only requires a modification to the traditional imaging model, it lends itself to compact and inexpensive implementations. It can therefore be incorporated into a wide array of devices, ranging from SLR to mobile phone cameras.