Trapping Light for Time of Flight

We propose a novel imaging method for near-complete, surround, 3D reconstruction of geometrically complex objects, in a single shot. The key idea is to augment a time-of- flight (ToF) based 3D sensor with a multi-mirror system, called a light-trap. The shape of the trap is chosen so that light rays entering it bounce multiple times inside the trap, thereby visiting every position inside the trap multiple times from various directions. We show via simulations that this enables light rays to reach more than 99.9% of the surface of objects placed inside the trap, even those with strong occlusions, for example, lattice-shaped objects. The ToF sensor provides the path length for each light ray, which, along with the known shape of the trap, is used to reconstruct the complete paths of all the rays. This enables performing dense, surround 3D reconstructions of objects with highly complex 3D shapes, in a single shot. We have developed a proof-of-concept hardware prototype consisting of a pulsed ToF sensor, and a light trap built with planar mirrors. We demonstrate the effectiveness of the light trap based 3D reconstruction method on a variety of objects with a broad range of geometry and reflectance properties.

Publications

"Trapping Light for Time of Flight,"
R. Xu, M. Gupta and S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
pp. 1-8, Jun. 2018.
[PDF] [bib] [©]

Images

  High Level Idea:

A collimated light source emits a light ray into a light trap made of mirrors. (a-c) Paths traced by the ray after 5 bounces, 50 bounces and 500 bounces, respectively. After a sufficient number of reflections, nearly all locations in the trap are visited multiple times from different directions

     
  Simulation for Testing Surface Coverage:

Here we provide a simulation result based on the high level idea. Figure (a) shows the simulation setup. A 4 by 4 by 4 lattice of spheres is placed in a cubic light trap. (b) Initial state of the simulation. (c) As the simulation runs, a color code is assigned to points on the surface of spheres, depending on the number of bounces the light ray undergoes before hitting the points. (d) End result of the simulation. (e) 99.9% of the lattice's surface area is visited after 5 bounces.

     
  3D Imaging Using Light Trap ToF Sensor:

A diffuse object is placed inside a light trap. A light ray emitted by a ToF sensor at p0 reaches scene point p after multiple bounces. The ToF sensor measures the total length of the light path from p0 to p, which, along with known shape of the trap, can be used to determine the 3D coordinates of p.

     
  Hardware Prototype of Light Trap Based 3D Imaging System:

A prototype of the Light Trap Based 3D Imaging System is as shown in the figure. (a-b) Design of the light trap and the supporting frame. (c) Complete setup with the ToF sensor (left), light trap and an object of interest placed on the object holder through the cutout of the trap (middle), and the additional 45-degree planar mirror (right)

     
  Experimental Results:

The end results obtained from the Light Trap Based 3D Imaging System is a dense, surround 3D reconstruction of the target object. In each row, we show, from the left to right, (1) a photo of the scanned object, (2) ToF camera view, (3) "raw" depth map recovered by the ToF camera, (4-5) two different perspective views of the reconstructed object.

     

Video

  CVPR 2018 Video:

This video introduces the Light Trap Based 3D Imaging System, illustrates the high-level idea, summarizes the reconstruction algorithm, and demonstrates several experimental results in detail. (With narration)