Face Modeling for Interaction

An alternative visual input is also being evaluated. Face modeling can be used to recover subtle facial detail (beyond blob tracking) for a more convincing interaction. A system which automatically detects the face and tracks it has been implemented [26]. It is capable of tracking the 3D rotations and movements of a face using normalized correlation coupled with structure from motion. In addition, at each moment in time, it computes an eigenspace model of the face's texture. This texture description is used to infer corresponding 3D deformations statistically [27]. This system generates a real-time temporal sequence which includes XYZ translations, 3D rotations as well as a set of texture and deformation scalar values (in an eigenspace). Figure 10.1 depicts the face tracking algorithm and examples of the temporal sequences being output in real-time.

  

Figure 10.1: 3D Face Modeling and Tracking

To synthesize an output, a 3D renderer reconstructs a 3D facial model in real-time using estimated deformation coefficients, texture coefficients, rotations and translations. The sample output is shown in Figure 10.1(d). The data representing each static frame can again be a time series (with 50 dimensional features) and the above ARL system analysis is currently being applied to this platform.