next up previous
Next: Summary and Conclusions Up: DyPERS: Dynamic Personal Enhanced Previous: Scenarios

A Sample Test Scenario

Evidently, DyPERS has many applications and it is unlikely to evaluate its performance in all possible situations. A usability study in a sample environment was selected to gain insight on real-world performance of the system as a whole. Since the system features audio-visual memory and significant automatic computer vision processing, test conditions involved these aspects in particular.

DyPERS was evaluated in a museum-gallery scenario. Audio-only augmented reality in a museum situation was previously investigated by [Bederson, 1995]. The museum constitutes a rich visual environment (paintings, sculptures, etc.) which is accompanied by many relevant facts and details (usually from a guide or text). Thus, it is an audio-visual educational experience and well-suited for verifying the system's usefulness as an educational tool.

A small gallery was created in our lab using 20 poster-sized images of various famous works ranging from the early 16th century to contemporary art. Three classes of human participants (types A, B, and C) were tested in a walk-through of the gallery while a guide was reading a script describing the paintings. The guide presented biographical and stylistic information about each painting while the subjects either used DyPERS (group A), took notes (group B) or simply listened attentively (group C). The subjects knew that they would be tested after the tour.

After the completion of the tour, the subjects were given a 20-question multiple-choice test containing one query per painting presented. In addition, the users had visual access to the paintings since these were printed on test sheets or still visible in the gallery. Thus, the subjects could refer back to the images while being tested. For each test session, subjects of all three types described above were present and examined (i.e. A, B, and C were simultaneously present and, thus, variations in the guide's presentation do not affect their relative performance). Table 1 contains the accuracy results for each of the user groups. The results suggest that the subjects using DyPERS had an advantage over subjects without any paraphernalia or with standard pencil and paper notes. Currently, arrangements are being made with the List Visual Arts Center2 for attempting the above test in their publically accessible contemporary art gallery.


 
Table: Subject Classes Accuracy
Group Description Range Avg. Accuracy
A DyPERS 90%-95% 92.5 %
B Note Pad 75%-95% 83.8%
C No Aid 65%-95% 79.0%
 



next up previous
Next: Summary and Conclusions Up: DyPERS: Dynamic Personal Enhanced Previous: Scenarios
Tony Jebara
1998-10-07