Motion and Transparent Objects |
 | The perception of transparent objects from images is known to be a very
hard vision problem. Given a single image, it is difficult to even detect the
presence of transparent objects in the scene. In this project, we explore what
can be said about transparent objects by a moving observer. We show how
features that are imaged through a transparent object behave differently from
those that are rigidly attached to the scene. We present a novel model-based
approach to recover the shapes and the poses of transparent objects from known
motion. The objects can be complex in that they may be composed of multiple
layers with different refractive indices. We have applied our algorithm to real
scenes that include transparent objects and recovered the shapes and dimensions
of the objects with high accuracy. |
Publications
"What Does Motion Reveal about Transparency?,"
M. Ben-Ezra and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
Vol.2, pp.1025-1032, Oct, 2003.
[PDF] [bib] [©]
"Transparent Grippers for Robot Vision,"
A. Nikolaev and S.K. Nayar,
IEEE International Conference on Robotics and Automation (ICRA),
Vol.2, pp.1644-1649, Apr, 1996.
[PDF] [bib] [©]
|
Images
 |
|
Refraction through Transparent Objects:
A ray of light interacts with a transparent object at two interfaces. The
first interaction occurs when the ray passes from the surrounding medium having
a refractive index of u1 into the transparent object having a refractive index
u2. The second interaction occurs when the ray passes from the transparent
object back into the surrounding medium. At each interface a portion of the ray
is reflected and another portion is refracted.
|
| |
|
|
 |
|
Recovery Method:
The set of parallel incident rays from a distant feature are scattered by a
transparent object in various directions. Some of these rays are captured by a
moving camera, when they pass through the optical center of the camera. Given
the intrinsic parameters and motion of the camera, we can compute the location
(in 3D) of each captured ray in space. Note that if we knew the shape and pose
of the object, these rays could be traced backwards to form a set of (outgoing)
parallel rays. This holds true regardless of the complexity of the transparent
object. It also remains valid for rays that go through total internal
reflections (like the one imaged by the camera in the figure) as well as rays
that do not pass through the transparent object (such as the bottom most
one).
|
| |
|
|
 |
|
Initial Guess:
The initial guess shape overlaid on one of the images. The initial guess was a
sphere (n=1.0).
|
| |
|
|
 |
|
The Cube:
An image of the actual cube used in the experiment shown to convey its shape.
Its refractive index u is unknown.
|
| |
|
|
 |
|
Computed Shape:
The recovered shape of the cube (n=0.18).
|
| |
|
|
 |
|
Initial Guess:
The initial guess shape is overlaid on one of the images. The initial diameter
of the cylinder was 200mm and the initial thickness of its wall was 20mm.
|
| |
|
|
 |
|
The Cylinder:
An image of the actual acrylic cylinder. It has a diameter of 117mm and wall
thickness of 3mm.
|
| |
|
|
 |
|
Computed Shape:
The computed diameter of the cylinder is 116.1mm and the computed thickness of
its wall is 2.3mm. The results are accurate even though this is a very complex
case with 4 interfaces.
|
| |
|
|
|
Videos
If you are having trouble viewing these .mpg videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.
 |
|
Transparent Cube and Moving Camera:
This video clip shows the nine views used to recover the shape of a
transparent cube with rounded edges.
|
| |
|
|
 |
|
Pipe Video:
This video clip shows the nine views used to recover the shape of an acrylic
cylinder filled with water (4 interfaces).
|
| |
|
|
|
Slides
ICCV 2003 presentation With videos (zip file)
|
Dirty Glass: Contamination on Transparent Surfaces
|
|
|
