LSA 2017: Intonation and Computation

Julia Hirschberg

Time: M/Th 3:30-5:20

Location: JSC 108
Prerequisite: None


Background reading: 

Hirschberg, “Pragmatics and Prosody”, Oxford Handbook of Pragmatics, ed. Y. Huang, Oxford University Press, 2017, Chapter 28.

Course readings are linked from this syllabus for each class; those identified by ‘*’ are optional readings.

Other suggested readings:  Keith Johnson. Acoustic & Auditory Phonetics (3rd edition). Wiley.  2011.



Praat: (tutorials:


Grade Breakdown

Discussion Questions; Paper Presentation (35%)

Students will be expected to complete all assigned readings before class, prepare 3 discussion questions related to the readings and bring to class to hand in at the end of class, and to actively participate in class discussions. Each class one student will prepare a short presentation on one of the optional readings to summarize the main points and critique the paper.

Project Proposal (65%)

Each student will prepare a proposal for an experiment or other corpus-based study of some topic involving one of the topics discussed in class or another type of speaker state.  The proposal should describe the question(s) to be asked; any previous work on the topic; how they would go about investigating the topic; what data they would make use of/collect; what obstacles they might encounter and how they would address them.  The proposal should be 5-7pp with extra pages for references as needed.

Academic Integrity
The SEAS academic integrity policy is found here.
The CS academic integrity policy is found here.

Note: Schedule and readings are subject to change





Assignments and Optional Readings

Day 1


Analyzing Speech Prosody

Modeling prosody; ToBI Conventions; Prosody and Meaning


Day 2


Distinguishing Deceptive from Non-Deceptive Speech and Text

Combining Acoustic-Prosodic, Lexical, and Phonotactic Features for Automatic Deception Detection
The Detection of Deception: The Effects of First and Second Language on Lie Detection Ability
Personality Factors in Human Deception Detection: Comparing Human to Machine Performance

Lying Words: Predicting Deception from Linguistic Styles

*Experiments in Open Domain Deception Detection
*Finding Deceptive Opinion Spam by Any Stretch of the Imagination

Try recording some speech with different contours and emotions in Praat

Day 3


Recognizing Emotional Speech and Text

Classifying Subject Ratings of Emotional Speech Using Acoustic Features
Predicting Student Emotions in Computer-Human Tutoring Dialogues
Using Context to Improve Emotion Detection in Spoken Dialog Systems

*Using Hashtags to Capture Fine Emotion Categories from Tweets  (Kyungnan Kim)

)Day 4


Charisma, Likability and Style in Speech and Text

Charisma perception from text and speech
"Would You Buy A Car From Me?"-- On the Likability of Telephone Voices (Samantha Cornelius)

*Extracting Social Meaning: Identifying Interactional Style in Spoken Conversation (Ty Slobe)

Day 5


Identifying Personality from Speech and Text

Automatic Recognition of Personality in Conversation
Automatically Classifying Self-Rated Personality Scores from Speech

*Computer-based personality judgments are more accurate than those made by humans

(Jacob Pawlak) Hand in 2-3pp draft of your proposal

Day 6


Sarcasm Detection in Speech and Text

Sarcastic or Not: Word Embeddings to Predict the Literal or Sarcastic Meaning of Words
"Sure, I did the right thing": A system for sarcasm detection in speech (Rachel Albar)
"Yeah, right": Sarcasm recognition for spoken dialogue systems (Colette Feehan)


Day 7

Mental Illness

Diagnosing Mental Illness from Speech and Text

Towards Automatically Classifying Depressive Symptoms from Twitter Data for Population Health
Vocal-Source Biomarkers for Depression: A Link to Psychomotor Activity
Efficacy of a Web-Based, Crowdsourced Peer-To-Peer Cognitive Reappraisal Platform for Depression: Randomized Controlled Trial

*Detecting late-life depression in Alzheimer's disease through analysis of speech and language (Betsy Miller)
*Towards Automatic Detection of Abnormal Cognitive Decline and Dementia Through Linguistic Analysis of Writing Samples (Chang He)
*Using linguistic features longitudinally to predict clinical scores for Alzheimer's disease and related dementias (Yuanda Liao)

Day 8


Who Do You Trust?

Detecting the Trustworthiness of novel partners in economic exchange
A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction 

*Trust and Deception in Mediated Communication (Vivian Li)

Describe your proposed experiment to the class in 7m

Turn in your full proposal by the end of the institute