Shape from Focus

Focus analysis provides a powerful means of recovering the shapes of visibly rough surfaces. These are surfaces with complex roughness and reflectance properties and are difficult, if not impossible, to recover using other vision techniques. In this project, we have developed a shape from focus method that uses different focus levels to obtain a sequence of object images. A focus operator is used to provide local measures of the quality of image focus. The operator is applied to the image sequence of the object to determine a set of focus measures at each image point. A model is developed that describes the variation of the focus measure due to defocusing. This model is used by a depth estimation algorithm to interpolate a small number of focus measure values and obtain accurate depth estimates. A fully automated shape from focus system has been implemented using an optical microscope and tested on a variety of industrial and biological samples.

Shape from focus relies on surface texture for the computation of depth. In many real-world applications, surfaces can be smoothly shaded and lacking in detectable textures. In such cases, shape from focus can generate inaccurate and sparse depth maps. We have extended the original shape from focus method to make it applicable to surfaces that have weak or no texture. A strong texture is forced on the imaged surface by the use of active illumination. The exact pattern of the projected illumination is determined through a Fourier analysis of all the optical effects involved in focus analysis. When the focus operator used is a 2-D Laplacian, the optimal illumination pattern is found to be a checkerboard whose pitch is the same size as the distance between adjacent elements in the discrete Laplacian kernel. This analysis also reveals the exact number of images required for accurate shape recovery. These results are experimentally verified using an optical microscope. Surfaces lacking in texture, such as three-dimensional structures on silicon substrates and solder joints on circuit boards, were used in the experiments. Experiments were also done using cells and plant samples.

Publications

"Are Textureless Scenes Recoverable?,"
H. Sundaram and S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
pp.814-820, Jun, 1997.
[PDF] [bib] [©]

"Focus Range Sensors,"
S.K. Nayar, M. Nigochi, M. Watanabe and Y. Nakagawa,
International Symposium on Robotics Research (ISRR),
Oct, 1995.
[PDF] [bib] [©]

"Microscopic Shape from Focus Using Active Illumination,"
M. Noguchi and S.K. Nayar,
IAPR International Conference on Pattern Recognition (ICPR),
Vol.A, pp.147-152, Oct, 1994.
[PDF] [bib] [©]

"Shape from Focus,"
S.K. Nayar and Y. Nakagawa,
IEEE Transactions on Pattern Analysis and Machine Intelligence,
Vol.16, No.8, pp.824-831, Aug, 1994.
[PDF] [bib] [©]

"Shape from Focus System,"
S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
pp.302-308, Jun, 1992.
[PDF] [bib] [©]

"Shape from Focus: An Effective Approach for Rough Surfaces,"
S.K. Nayar and Y. Nakagawa,
International Conference on Robotics and Automation,
Vol.1, pp.218-225, May, 1990.
[PDF] [bib] [©]

Image

  Hitachi-Siemens Depth Scanning System:

Several of the techniques developed as a part of this project were used to develop this high-speed depth scanning system. The system was used to inspect Hiatchi's MP5800 Large Scale Computer. Courtesy of Hitachi's Production Engineering Research Laboratory.
     

Videos

If you are having trouble viewing these videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.

  Shape from Focus System:

This video shows a microscope-based shape from focus system. The stage of an off-the-shelf optical microscope was motorized to allow computer-controlled sample positioning in the z-direction. In a second version of the system, an analytically derived illumination pattern is projected onto the sample to force a dominant single-frequency texture. A small set of images (10-20) is taken as the sample is displaced in steps towards the objective lens and a depth map and all-focused image of the sample is computed.
     
  Silicon Wafer Example:

This video shows the computed three-dimensional texture-mapped surface of rectangular structures on a silicon wafer. The height of the structures is approximately 13 microns.

     
  Leaf Structure:

This video shows the computed structure of a section of a green leaf. In the middle of the structure is a stomata (air vent) that is approximately 30 microns in height.
     

Depth from Defocus

Generalized Mosaicing

Shape from Brightness