Active Refocusing of Images and Videos |
 | In this project, we present a system for refocusing images and videos of
dynamic scenes using a novel, single-view depth estimation method. Our method
for obtaining depth is based on the defocus of a sparse set of dots projected
onto the scene. In contrast to other active illumination techniques, the
projected pattern of dots can be removed from each captured image and its
brightness easily controlled in order to avoid under- or over-exposure. The
depths corresponding to the projected dots and a color segmentation of the
image are used to compute an approximate depth map of the scene with clean
region boundaries. The depth map is used to refocus the acquired image after
the dots are removed, simulating realistic depth of field effects. Experiments
on a wide variety of scenes, including close-ups and live action, demonstrate
the effectiveness of our method. |
Publications
"Active Refocusing of Images and Videos,"
F. Moreno-Noguer, P.N. Belhumeur and S.K. Nayar,
ACM Trans. on Graphics (also Proc. of ACM SIGGRAPH),
Aug, 2007.
[PDF] [bib] [©]
|
Images
 |
|
Active Refocusing Steps:
(a) Image acquired by projecting a sparse set of illumination dots on the
scene. (b) The dots are automatically removed. (c) The defocus of the dots is
used to compute a sparse depth map. (d-f) Color segmentation of the image is
used to compute a complete depth map of the scene with sharp boundaries. (g-i)
The depth map and the dot-removed image are used to smoothly refocus the scene.
(j) The refocusing can also be done for an image taken immediately before or
after, with new lighting.
|
| |
|
|
 |
|
Controlling the Brightness of the Projected Dots:
Top: Images of a square dot projected onto different depths. Bottom: The
radiance Iw of a projected dot may be controlled by changing the parameters of
the setup, such as the distance uf of the focal plane, or the with w of the
projected dots. By doing this, we can avoid under- or over-exposure, and ensure
that the brightness of all the points in the image are within the dynamic range
of the camera.
|
| |
|
|
 |
|
Realistic Refocusing:
In order to render realistic depth of field effects we propose an algorithm which considers partial occlusions (for object boundaries, different parts of the lens may see different views). Left: original all-focused image. Right: refocusing results obtained with the proposed algorithm, where the virtual focal plane is placed on the background of the scene. Observe that the results are highly realistic, especially at the depth discontinuities.
|
| |
|
|
|
Video
If you are having trouble viewing these .mov videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.
 |
|
SIGGRAPH 2007 Video:
This video shows all our examples of image and video relighting based on the
defocus of a sparse set of dots projected onto the scene. (With narration)
|
| |
|
|
|
Projection Defocus Analysis
Depth from Defocus
Shape from Focus
Projection Defocus Analysis
|
|
|
