The formation of an out-of-focus image (f) can be written as a convolution of the focused image (f_0) by the point spread function (k) plus image noise (xi). It is well known that a conventional circular aperture results in a point spread function that severely attenuates high frequencies and has many zero-crossings in the frequency domain. As a result some frequencies simply cannot be recovered by deconvolution and noise in the deblurred image is greatly exaggerated by the process of deconvolution.

There are many possible aperture designs, each giving a different profile in the frequency domain. If the point spread function has low response at a certain frequency then the recorded image will have a lower signal-to-noise ratio, and the deblurred image will as well. Because of this, one may want a pattern which has high responses everywhere in the frequency domain. Unfortunately, the frequency response of a point spread function must have finite overall energy, and therefore cannot be high everywhere. Then, how should we measure the pattern optimality?

In this project, we will not assess aperture patterns directly based on the shape of the power spectra. Instead, we are going to propose an evaluation criterion R(K) which reflects the expected quality of the recovered image when the pattern (K) is used. To be more specific, in our framework, the input image (F_0) and image noise (eta) are regarded as two random vectors, and R(K) is defined as the expectation of the distance between the ground truth focused image and the recovered image.

Note that the expectation of deblurring quality is related to the distribution of the latent focused image, the noise model, the deblurring algorithm, and the measurement of image quality. To make the problem more tractable, we assume a distribution of natural images, Gaussian white noise, optimal linear deblurring algorithm, and use L2 distance as a quality measurement.

A concise and comprehensive evaluation criterion of aperture patterns has been derived. Given the Fourier transform of any PSF, K, the proposed function R(K) indicates the expectation of its deblurring performance. The lower the R value is, the better the deblurring performance. Two observations can be made by analyzing this formulation. One, the optimality of a pattern is highly related to the noise level. Two, different frequencies are not equally important for deblurring.

The proposed criterion was used to evaluate many existing aperture patterns at various noise levels. Our evaluation shows that the conventional circular aperture tends to be superior when image noise level is high, while coded apertures (such as our "Image pattern" and Veeraraghavan et al.'s pattern) can do much better when image noise level is low.

Even with our concise evaluation criterion, finding the optimal aperture pattern remains a challenging problem. While the aperture pattern is evaluated in the frequency domain, it must satisfy several physical constraints in the spatial domain. We therefore resort to a numerical search approach and optimize the pattern using a genetic algorithm. Experiments show that for patterns of resolution 13 x 13 or less, the genetic algorithm can converge quickly.

The proposed genetic algorithm is compared with a randomized linear search, commonly used in previous work. Each algorithm is repeated 10 times. We see that our genetic algorithm converges quickly to a low value for aperture criterion metric. Also, the results of different runs of the genetic algorithm are quite similar, indicating that they may all lie close to the optimum aperture.

We present the optimized patterns for eight different noise levels and their quality measured by our criteria. It is interesting to note that the optimized aperture patterns get more structured with increase in noise.

We printed our optimized aperture patterns as well as several other patterns in high resolution (1 micron) on a photomask sheet. To experiment with a specific aperture pattern, we inserted it into a Canon EF 50mm f=1.8 lens.

We placed a CZP resolution chart at the distance of 150cm from the lens, and captured images using the five different apertures: circular pattern, our "image pattern", the pattern from Levin et al., the pattern from Veeraraghavan et al., and one of our optimized patterns. The same exposure time was used for all the acquisitions. Note that our optimized pattern gives the sharpest deblurred image with least artifacts and image noise.

On the left is an out-of-focus Shrek image captured using the optimized aperture pattern, while on the right is the deblurring result.

On the left is an out-of-focus street scene captured using the optimized aperture pattern, while on the right is the deblurring result. Although some ringing artifacts can be seen in our deblurred images, significant details are recovered by the use of our optimized pattern. Note that in the captured image, the degree of defocus is so great that many scene details are virtually invisible.