Each row shows training examples of face images that match the given attribute label (positive examples) and those that don't (negative examples). We have over a thousand training images for each of our 65 attributes. Accuracies for each attribute classifier are shown in the next image.

We present accuracies of the 65 attribute classifiers trained for our system. Example training images for the attributes in bold are shown in the previous images

We use Amazon Mechanical Turk to label images with attributes. This online service allows us to easily and inexpensively label images using large numbers of human workers. This image shows an example of our attribute labeling jobs. We were able to collect over 125,000 human labels in a month, for $5,000.

An attribute classifier can be trained to recognize the presence or absence of a describable aspect of visual appearance. The responses for several such attribute classifiers are shown for a pair of images of Halle Berry. Note that the "flash" and "shiny skin" attributes produce very different responses, while the responses for the remaining attributes are in strong agreement despite the changes in pose, illumination, expression, and image quality.

Each simile classifier is trained using several images of a specific reference person, limited to a small face region such as the eyes, nose, or mouth. We show here three positive and three negative examples for four regions on two of the reference people used to train these classifiers.

We use a large number of "simile" classifiers trained to recognize the similarities of parts of faces to specific reference people. The responses for several such simile classifiers are shown for a pair of images of Harrison Ford. R_j denotes reference person j, so the first bar on the left displays the similarity to the eyes of reference person 1. Note that the responses are, for the most part, in agreement despite the changes in pose, illumination, and expression.

Performance of our attribute classifiers, simile classifiers, and a hybrid of the two are shown in solid red, blue, and green, respectively. All 3 of our methods outperform all previous methods (dashed lines). Our highest accuracy is 85.29%, which corresponds to a 31.68% lower error rate than the previous state-of-the-art.

We asked human users on Amazon Mechanical Turk to perform the face verification task on the LFW data set. This image shows an example of what these jobs looked like. Using a total of 240,000 user responses, we were able to plot human performance on LFW

Human performance on LFW is almost perfect (99.20%) when people are shown the original images (red line). Showing a tighter cropped version of the images (blue line) drops their accuracy to 97.53%, due to the lack of context available. The green line shows that even with an inverse crop, i.e., when only the context is shown, humans still perform amazingly well, at 94.27%. This highlights the strong context cues available on the LFW data set. All of our methods mask out the background to avoid using this information.

We show example images for the 140 people used for verification tests on the PubFig benchmark. Below each image is the total number of face images for that person in the entire data set.

Our performance on the entire benchmark set of 20,000 pairs using attribute classifiers is shown in black. Performance on the pose, illumination, and expression subsets of the benchmark are shown in red, blue, and green, respectively. For each subset, the solid lines show results for the "easy" case (frontal pose/lighting or neutral expression), and dashed lines show results for the "difficult" case (non-frontal pose/lighting, non-neutral expression).

As a complement to the LFW data set, we have created a data set of images of public figures, named PubFig. This data set consists of 60,000 images of 200 people. The larger number of images per person (as compared to LFW) allows us to construct subsets of the data across different poses, lighting conditions, and expressions, while still maintaining a sufficiently large number of images within each set.