Temporal Modulation Imaging |
 |
In this project, we are interested in the capture of high dynamic range and
multispectral images of complex scenes. Our approach is to capture multiple
frames of the scene while the imaging parameters are modulated continuously;
images are taken at different exposures or through different wavelength bands.
A major problem associated with such a continuous modulation approach is the
need for perfect synchronization between image acquisition and modulation
control. In the past, this problem has been addressed by using sophisticated
servo-control mechanisms. In this work, we show that the process of modulation
imaging can be made much simpler by using vision algorithms to automatically
relate each acquired frame to its corresponding modulation level. This
correspondence is determined solely from the acquired image sequence and does
not require measurement or control of the modulation. The image acquisition and
the modulation work continuously, in parallel, and independently. We refer to
this approach as computational synchronization. It makes the imaging process
simple and easy to implement. We have developed a prototype modulation imaging
system that uses computational synchronization and we have used it to acquire
high dynamic range and multispectral images. |
Publications
"Uncontrolled Modulation Imaging,"
Y.Y. Schechner and S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol.II, pp.197-204, Jun, 2004.
[PDF] [bib] [©]
|
Images
 |
|
Modulation Imaging System:
This system was used for demonstrating the method of uncontrolled modulation
imaging. It uses a monochrome image detector. Adjacent to the detector (before
the lens), there are two spatially varying filters: one with varying
transmittance (the circular filter), the other with a varying spectral response
(the rectangular filter).
|
| |
|
|
 |
|
Exposure Modulation for Scene with Person:
This is the captured video for a scene with a person captured by the system
for HDR imaging. Notice the continuous change in exposure.
|
| |
|
|
 |
|
HDR Image of Scene:
This is the high dynamic range image computed from the above video clip.
Notice how the whole scene is measured without any dark and saturated regions.
This image was obtained by dynamic range compression of the computed HDR image
to show all the details.
|
| |
|
|
 |
|
Spectrum Modulation for Flower Scene:
This is the captured video for a scene with a bunch of flowers captured by the
system for multispectral imaging. Notice the continuous change in measured
spectrum which is shown here as a change in RGB color. Notice how the colors
change for the different types of flowers.
|
| |
|
|
 |
|
Multispectral Image of Flowers:
This is the multispectral image computed from the above video for the flower
scene. Here the measured spectrum at each pixel is mapped to an RGB color value
(for visualization purposes) based on its peak location within the visible
spectrum.
|
| |
|
|
 |
|
Spectrum Modulation for Bulbs:
This is the captured video for a scene with three different types of light
bulbs captured by the system for multispectral imaging. The scene includes an
incandescent bulb (left), a halogen bulb (middle) and a fluorescent bulb
(right). Notice how the bulbs peak (sometimes multiple times) at different
points during the scanning.
|
| |
|
|
 |
|
Multispectral Image of Bulbs:
This is the multispectral image computed from the above video for the bulb
scene. Here the measured spectrum at each pixel is mapped to an RGB color value
(for visualization purposes) based on its peak location within the visible
spectrum.
|
| |
|
|
|
Videos
If you are having trouble viewing these .mpg videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.
 |
|
HDR Imaging:
This video shows the capture of images during continuous change of exposure
(transmittance). The exposure varies as the circular spatially varying neutral
density filter rotates.
|
| |
|
|
 |
|
Multispectral Imaging:
This video shows the capture of images during continuous change of spectral
filtering. The spectral response of the camera varies as the linear
interference filter translates back and forth.
|
| |
|
|
 |
|
Multidimensional Modulation Imaging:
In this video the neutral density filter and the linear interference filter
are both simultaneously in motion. If one filter completes a large number of
modulation cycles for a single cycle of the other filter, then a high dynamic
range multispectral image can be computed.
|
| |
|
|
|
Slides
ICCV 2005 presentation With videos (zip file)
|
Database
WILD: Weather and Illumination Database
|
High Dynamic Range Imaging: Assorted Pixels
High Dynamic Range Imaging: Multiple Exposures
Radiometric Camera Calibration
Generalized Mosaicing
Coded Rolling Shutter Photography:
Flexible Space-Time Sampling
|
|
