Imaging of objects under variable lighting directions is an important and frequently used procedure in computer vision and computer graphics. In this project, we have developed an approach that significantly improves the quality of images under different lighting directions. Traditional methods for acquiring images under different illumination directions use only a single light source per acquired image. In contrast, our approach is based on the principle of multiplexing in which multiple light sources illuminate the object simultaneously from different directions. Thus, the object irradiance is higher. The acquired images are then computationally demultiplexed. This approach does not increase the number of image acquisitions. It is useful for imaging dim object areas and highly specular objects. For cameras having signal-independent noise, the optimal multiplexing is achieved by using Hadamard codes, yielding significantly improved signal-to-noise ratio in the demultiplexed images. This advantage may be translated into faster acquisition times or an increase in density of illumination direction samples. It also enables one to use lighting with high directional resolution, which is easy to achieve using a digital projector as we demonstrate in our experiments. Moreover, the multiplexing approach enables imaging of diffuse reflectance while avoiding saturation of specular highlights. It is also more robust to stray light. We have analyzed limitations of the approach and identified scenarios in which the approach should be avoided.

Publications

"A Theory of Multiplexed Illumination,"
Y.Y. Schechner, S.K. Nayar and P.N. Belhumeur,
IEEE International Conference on Computer Vision (ICCV),
Vol.2, pp.808-815, Oct, 2003.
[PDF] [bib] [©]

Images

		Cup and Bust: This image shows the benefits of using multiplexed illumination. The bottom half of the image shows the poor quality of an image of the scene acquired with a single source. The top half shows the same single source image that was computed by demultiplexing the set of images acquired using multiplexed illumination. Note the high signal-to-noise ratio of the image. Furthermore, the specularities on the cup and the bust are well preserved and not saturated.
		Single-Source Images of a Set of Diffuse Objects: These are two images of a set of diffuse objects taken under single-source lighting. In this case, each light source was created by projecting a small white patch on a screen. The raw frames are very dark. Amplification of these images reveal their very low signal-to-noise ratio.
		Demultiplexed Single-Source Images: These are demultiplexed single-source images that were computed from an image sequence of the above diffuse objects taken using multiplexed illumination. One can see that these images are of much higher quality than the raw single-source images shown in the above picture.

Video

		Multiplexed Illumination Using a Projector: This video shows the setup we have used in our multiplexing experiments. Light patterns are projected on a screen. These patterns illuminate the objects of interest. The patterns in the movie are Hadamard codes. They yield high irradiance of the objects (resulting in high signal-to-noise ratio) while avoiding saturation of specular highlights.

Slides

ICCV 2003 presentation     With videos (zip file)

Related Project

Optimal Illumination for Video Relighting