One more degree of freedom in our 7 parameter set, sy has yet to be set. To determine the optimal value of this parameter, we shall now consider the position of the mouth on our model m3 and on our image i3. The other vertices (eyes and nose) have been exactly mapped to their destinations on the image via the WP3P computation. However, the same is not true for the mouth whose locus in the image has had no bearing so far on the calculation of our WP3P solution. In fact, we expect an error, Emouth in the projected model mouth locus (mp3) and the image's mouth locus (i3). Emouth is the distance between the model's mapped mouth and the image's mouth as shown in Equation :
To minimize Emouth, we solve the WP3P for several iterations of sy. We try the following values of sy in attempting to find the best fit to the face: . The model and its four anchor points are scaled iteratively in the y dimension by each value of sy, and the WP3P is solved. The WP3P solution which generates the smallest error, Emouth, is then found corresponding to the best sy parameter. We have thus solved the problem of fitting to the mouth point by allowing our model to stretch along y to fit to ``squashed'' and elongated faces. Incidentally, since only a simple vertical stretching is involved in this minimization, it is evident that only the perpendicular distance of the mouth from the eyes will affect the deformation and not the horizontal displacement of the mouth. No matter where the mouth is horizontally, the model fitting will remain the same. Figure depicts a few sample stretches to find the best model to fit the image. The texture on the face is a mesh representing the frontal face of the 3D model. The mesh undergoes multiple stretches and is aligned to the eyes and nose each time with the WP3P computation. The stretched mesh whose mouth point best aligns with the mouth point in the image is then used as the 3D structure which will be coated with the face in the intensity image.
Computing the above total 4-point fitting yields the rotation, deformation and translation to apply to the 3D model to fit it to the image. Our implementation is also highly computationally efficient, requiring less than 1 millisecond on an SGI Indy workstation.