In this project, we pursue a radically different approach to imaging. Rather than seeking to
capture the world from a single point in space, our goal is to explore the idea of imaging
using a thin, large, flexible sheet. If such cameras can be made at a low cost (ideally, like a
roll of plastic sheet), they can be used to image the world in ways that would be difficult to
achieve using one or more conventional cameras. In the most general sense, such an imaging system would
enable any surface in the real world to capture visual information. While there is significant ongoing
work on the development of flexible image sensors, our interest here is in the design of the optics needed
to form images on such sensors.
At first glance one might imagine that a simple lens array aligned with a flexible detector
array would suffice - its field of view (FOV) can be varied by simply bending it. What is perhaps less
apparent is the fact that, in a curved state, the FOV can end up being severely under-sampled. This
under-sampling leads to a captured image that is not bandlimited. Thus, the Nyquist sampling criterion
is violated and the image will suffer from aliasing artifacts when reconstructed. It is important to note
that these artifacts cannot be removed via post-processing since scene information is lost during image formation.
To address the above aliasing problem over an entire range of sheet curvatures, we propose the
design of a deformable (elastic) lens array. We show that, if designed carefully, the deformable
lenses of the array will change shape (and hence focal length) under bending forces in a way that
mitigates aliasing. A remarkable feature of our design is that the lens array can achieve aliasing
compensation passively, without the use of any per-pixel actuation or control. Our optics can be
combined with a flexible sensor array to obtain a complete sheet camera. This project was supported by the
Office of Naval Research (ONR). The flexible lens array was prototyped using the facilities of The Columbia
Laboratory for Unconventional Electronics, led by John Kymissis.
This figure shows a lens array, where each lens has a fixed focal length, in a
highly curved state. Notice the large black areas (gaps) between the fields of
view (yellow regions) of adjacent lenses. These gaps lead to aliasing in the
captured image. We could attempt to eliminate the gaps by increasing the field
of view of each lens, but this would lead to blurring in the captured image
when the lens array is flat.
Adaptive Focal Length Lens Array:
Here, we see an elastic lens array where the focal length of each lens can adapt to local
curvature. Notice the contiguous coverage of the scene achieved by this lens array, with
no gaps between the fields of view of adjacent lenses. If the geometric and material properties
of the elastic lens array are designed well, this desired adaptation of focal length will hold
true over a wide range of curvatures of the array, avoiding aliasing and blurring in the captured image.
Passive Optical Adaptation to Local Curvature:
This graph shows the change in field of view of a single lens of the array as a function of the local curvature.
The black line (FOV des) shows the desired change in field of view of a single lens as the local curvature of
the lens array increases. The blue line (FOV act) is a plot of the field of view of a single adaptive focal
length lens in an elastic array as the local curvature increases. The red line (FOV fix) represents the FOV of
a fixed focal length lens, which does not change as a function of local curvature.
Effect of Material Properties on Optical Adaptation :
We modeled the deformable lens array using the Abaqus finite element analysis software suite. Using this model,
we studied how the material properties of the lens array affect its shape when it is deformed. We found that the
two properties that had the greatest impact were hardness (represented using the Shore A scale) and the
compressibility (represented by the Poisson's Ratio). In this video, we show four lens arrays of different material properties being deformed.
The colors within each lens array correspond to the mechanical stress. This simulation reveals that the shapes of the lenses of the array are more or
less independent of hardness, even as the mechanical stress differs. Furthermore, they are also independent of the Poisson's ratio as long as the
ratio is close to 0.5, which corresponds to an incompressible material.
Simulated PSF as a Function of Curvature:
Using Abaqus, we simulated how the lenses of the array we designed deform when the array is bent. Then, using a ray-tracing algorithm, we
simulated the object side point spread function (PSF) of the central lens of the array. As desired, the object-side PSF of that lens widens
as the local curvature increases. We have found that every lens, except those close to the edge of the array, behave in the same manner.
Aliasing in the Fixed and Adaptive Focal Length Cases:
In this figure we compare rendered images captured using fixed focal length and adaptive focal length lens array.
The scene texture is the sum of two sinusoids, one low and one high in frequency. Using the known optical properties
(PSFs and sampling period) of the two arrays, we rendered images captured by the two systems. For each curvature of
the arrays, we assumed that the scene surface is also curved with the same center of curvature as the arrays. Since
the FOV of the sheet camera increases with curvature, we interpolated the captured images in the horizontal direction
to obtain stretched images that depict the increasing horizontal FOV of the array. For the adaptive system, due to
blurring, the high frequency sinusoid decreases in magnitude with curvature, while the low frequency sinusoid is
faithfully reproduced over the entire range of deformations. In contrast, due to the narrow and fixed FOV of the
lenses in the non-adaptive case, the scene texture is under-sampled for the higher curvatures, which results in
undesirable aliasing artifacts.
Fabrication of the Elastic Lens Array:
This set of photos illustrate the procedure used to fabricate the elastic lens array. A 33 by 33 lens array
mold was machined from aluminum by Contour Metrological and Manufacturing, Inc. We prepared Silopren Liquid
Silicone Rubber 7005, a two-part solution, by mixing the two components together in a beaker and then
removing the air bubbles that form during mixing using a vacuum chamber. The mold is cleaned and then
coated with release agent to make it easy to remove the lens array from the mold after it is cured. We
then pour the silicone rubber into the mold and place the mold into an oven. Once the material is cured
(hardened) we remove the lens array from the mold. Next, a 33 by 33 aperture sheet made of nylon is glued to
the bottom of the lens array. We then attach a diffuser on which each lens forms a spot of light. The
entire grid spots formed by the lens array corresponds to a single captured image.
This photo shows the apparatus we have used to conduct our experiments. The deformation of the sheet camera is
controlled using a vise-like mechanism - the parallel jaws of the vise push against two sides of the imaging
sheet such that the distance between the jaws determines the curvature of the sheet. We used a Nikon D90 to
capture an image of the 33 by 33 spots created by the lens array on the diffuser attached to the bottom of the
array. An LCD monitor is placed above this apparatus to show various scenes to the imaging system.
Measured PSF as a Function of Curvature:
For each curvature of the lens array, we raster scanned a small bright dot on the LCD monitor above the apparatus.
As this dot was scanned, we measured the brightness of the spot on the diffuser produced by the central lens of
the array. The measured brightness values are used to plot the object side PSF of the central lens. As seen this
figure, the measured PSF increases in width as the local curvature increases, demonstrating that the FOV of the lens
does indeed increase with curvature to mitigate image aliasing.
Scenes Captured by the Flexible Sheet Camera:
We imaged various static scenes using the sheet camera. For each scene, we bent the array from the flat state
to a curved one for which the angular field of view is 52 degrees. For any given sheet deformation, we know the directions
of the optical axes of all the lenses of the array. This information is used to project the captured image onto a
plane at a chosen distance from the sheet camera. The projected measurements are then interpolated to obtain the
final image whose horizontal field of view increases with the curvature of lens array. Due to the inherent optical
adaptation of our design, the images are free of aliasing effects.
ICCP 2016 Video:
This video introduces the concept of flexible sheet cameras, discusses several potential applications of these flexible
cameras and summarizes the mechanism of passive focal length adaptation to local curvature. (With narration)