Frank Enos - Research

research

Deceptive Speech
A relatively limited amount of work has been done from a speech processing perspective to determine what acoustic and prosodic cues can reliably distinguish deceptive from non-deceptive speech; even less work has been done in the domain of automating this task. Most people perform very poorly at deception detection tasks, in general doing no better than chance.

Julia Hirschberg, Stefan Benus, and I, along with colleagues from SRI/ICSI (and formerly the University of Colorado at Boulder), are working on distinguishing deceptive from non-deceptive speech. Our ongoing work focuses on detection of deception by applying machine learning techniques to acoustic, prosodic, and linguistic features extracted from the Columbia-SRI-Colorado Deception Corpus. This corpus was collected at Columbia and represents a paradigm in which subjects lied to an interviewer with respect to their performance on a series of tests. The interviewer interrogates the subject with respect to their performance in the six test areas; subjects are directed to claim that their scores matched an ideal profile. Preliminary results using machine learning experiments are reported in our Eurospeech 2005 paper.

In addition, we have conducted a perception study that assesses human judges' ability to distinguish between deceptive and truthful speech in this corpus. Our intention here is both to establish a human baseline for our classification task, and to identify and examine subjects that are particularly easy or particularly difficult for humans to classify. We have also been working with Robin Cautin to discover attributes, such as personality factors, that make humans better (or worse) detectors of deception. Our results do, in fact, indicate that such factors exist.

Our other work on this corpus includes an examination of the use of filled pauses as a cue to deceptive speech, and the combination of multiple learners using different feature sets to create a single classifier.

Detecting Deception in Speech Frank Enos Ph.D. Dissertation January, 2009

Detecting Deception Using Critical Segments Frank Enos, Elizabeth Shriberg, Martin Graciarena, Julia Hirschberg, Andreas Stolcke In Proceedings of Interspeech 2007, Antwerp. Personality factors in human deception detection: Comparing human to machine performance Frank Enos, Stefan Benus, Robin Cautin, Martin Graciarena, Julia Hirschberg, Elizabeth Shriberg ICSLP 2006, Pittsburgh, PA. Pauses in Deceptive Speech Stefan Benus, Frank Enos, Julia Hirschberg, Elizabeth Shriberg Speech Prosody 2006, Dresden, Germany. Combining Prosodic, Lexical and Cepstral Systems for Deceptive Speech Detection M. Graciarena, E. Shriberg, A. Stolcke, F. Enos, J. Hirschberg, S. Kajarekar Proc. IEEE ICASSP, Toulouse, France, 2006. Distinguishing Deceptive from Non-Deceptive Speech Julia Hirschberg, Stefan Benus, Jason M. Brenier, Frank Enos, Sarah Friedman, Sarah Gilman, Cynthia Girand, Martin Graciarena, Andreas Kathol, Laura Michaelis, Bryan Pellom, Elizabeth Shriberg, Andreas Stolcke. In Proceedings of Eurospeech 2005. Related Work In May 2006, I gave an invited talk at the University of Bologna on our deception work: Il Rilevamento Automatico dell'Inganno nel Linguaggio Parlato. (The Automatic Detection of Deception in Speech.) I am also involved in a number of outside projects on deception. In the summer of 2005, I participated in the NSF's Workshop on Behavioral Aspects of Security, organized by Mark Frank of the University at Buffalo. In 2004, I participated in the Workshop on Detecting Deception in Language and Cultural Context at U.M.D.'s Center for the Advanced Study of Language (CASL). I have worked with CASL on a number of projects relating to deception in various cultures and languages. Here's another talk on deceptive speech. Emotional Speech I am also interested in the automatic recognition and generation of emotional speech in human-computer interaction. This work has a variety of applications, from the creation of intelligent interfaces that recognize the emotional state of the user, to decision-making systems that employ the sort of insight characteristic of human ‘reason’ (which is inherently emotional, according to a growing body of psychological research). In addition, recognition of emotion has the potential for direct application to natural language understanding, since it seems likely that the accuracy with which a hearer perceives the subjective state of the speaker affects the degree to which the hearer can perceive subtle (or not so subtle) shades of the speaker's meaning. My ongoing interests include the development of a model of emotion for use in this area, and I am particularly intrigued in the intersection of speaker intention with context, and how that intersection may generate what is perceived in speech (and otherwise) as emotion. At the LREC 2006 Workshop on Corpora for Research on Emotion and Affect I presented a paper written with Julia Hirschberg on a novel approach to eliciting emotional speech from actors using the methods employed in theater rehearsal. Unlike approaches that ask the actor to pretend to have the relevant emotion, this approach induces emotion by taking advantage of techniques developed in the professional theater for this purpose. This approach has the advantage of allowing the induction of a tremendous range of emotions, both positive and negative, in an ethical manner, since professional actors are, by virtue of their training and experience, accustomed to dealing with the induction of negative emotions. A framework for eliciting emotional speech: Capitalizing on the actor's process Frank Enos and Julia Hirschberg LREC 2006 Workshop on Corpora for Research on Emotion and Affect, Genova, Italy. Emotion Materials A presentation on emotional speech. An annotated bibliography on emotion in general and emotion and computation. Computational Linguistics and the Performing Arts My research interests also include the potential application of lessons from the performing arts process to the field of computational linguistics. This interest stems in part from my understanding of the performing arts as a transfer of empathy — that is, of subjective understanding. In addition, it is commonplace to find that an inexperienced actor can report what he is "feeling", but that a convincing actor is often more likely to report what he is "doing". Taken together, these aspects of theater provide a potential laboratory for the study of some of the phenomena described above.

Frank Enos • 726 CEPSR • 212.939.7122 • frank [æt] cs.columbia.edu