COMS 6998: Advanced Topics in Spoken Language Processing

Instructors: Julia Hirschberg and Sarah Ita Levitan

Time: F 10:10-12:00 (Spring 2019)

Location: Mudd 627


Prerequisite: COMS 4705 or another speech or NLP class

Description:  This class will introduce students to spoken language processing:  basic concepts, analysis approaches, and applications.

Required readings:

Jurafsky & Martin 2019 chapters

These and other readings are linked from this syllabus for each class.


Keith Johnson. Acoustic & Auditory Phonetics (3rd edition). Wiley.  2011.



A list of resources can be found here.


Office Hours

Julia Hirschberg: Friday, 12-1 (CEPSR 705)

Sarah Ita Levitan: Thursday, 11-12 (CEPSR 701)

Rose Sloan: Wednesday, 3-4 (CEPSR 7LW3 -- Speech Lab)

Zixiaofan (Brenda) Yang: Tuesday, 2-3 (CEPSR 7LW3 -- Speech Lab)

Grade Breakdown

5% attendance and participation

15% weekly posts

15% HW1

15% HW2

25% HW3

25% HW4



Academic Integrity

The SEAS academic integrity policy is found here.

The CS academic integrity policy is found here.


Note: Schedule and readings are subject to change






Week 1: 1/25

Introduction to Speech Processing


Week 2: 2/1

From Sounds to Language

Jurafsky & Martin Chapter 7 (sections 1-3)

Week 3: 2/8

Acoustics of Speech

Jurafsky & Martin Chapter 7 (sections 4-7)

Week 4: 2/15

Tools for Speech Analysis

Praat Tutorial(Chapter 11 - scripting - is optional)

Download Praat

HW1: Praat Recording and Analysis (assigned)

Week 5: 2/22

Analyzing Speech Prosody

ToBI Conventions

Modeling Prosody

Prosody and Meaning

HW1 due

Week 6: 3/1

Text-to-Speech Synthesis

Jurafsky & Martin Chapter 8

Merlin Tutorial

HW2: ToBI Labeling (assigned)

Week 7: 3/8

Speech Recognition: Then and Now

Jurafsky & Martin Chapter 9

Deng & Yu Chapter 7

Week 8: 3/15

Speech Analysis: Entrainment in Spoken Language
Guest lecture: Rivka Levitan
Guest lecture: Zhihua (Shirley) Xia
Entrainment in Deceptive Speech

Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions

Mark My Words! Linguistic Style Accommodation in Social Media

Prosodic entrainment in Mandarin and English: a cross-linguistic comparison

HW2 due

Week 10: 3/29

Spoken Dialogue Systems

Jurafsky & Martin Chapter 24

Jurafsky & Martin Chapter 25

HW3: Dialogue Acts (assigned)

Week 11: 4/5

Speech Analysis: Personality and Mental State

Detecting late-life depression in Alzheimer's disease through analysis of speech and language

Vocal-Source Biomarkers for Depresion: A Link to Psychomotor Activity

Automatic Recognition of Personality in Conversation

Week 12: 4/12

Speech Analysis: Deception and Trust

Linguistic Cues to Deception and Perceived Deception in Interview Dialogues

Lying Words: Predicting Deception from Linguistic Styles

Personality Factors in Human Deception Detection: Comparing Human to Machine Performance

HW3 due

Week 13: 4/19

Speech Analysis: Emotion, Sentiment and Keyword Search

Classifying Subject Ratings of Emotional Speech Using Acoustic Features

Using Context to Improve Emotion Detection in Spoken Dialog Systems

Adieu features? end-to-end speech emotion recognition using a deep convolutional recurrent network

HW4: Emotional Speech Detection (assigned)

Week 14: 4/26

Speech Analysis: Sarcasm and Humor

Sarcastic or Not: Word Embeddings to Predict the Literal or Sarcastic Meaning of Words

"Sure, I did the right thing": A system for sarcasm detection in speech

"Yeah, right": Sarcasm recognition for spoken dialogue systems

Week 15: 5/3

Speech Analysis: Charisma, Likability and Style

Charisma perception from text and speech

"Would You Buy A Car From Me?"-- On the Likability of Telephone Voices

Extracting Social Meaning: Identifying Interactional Style in Spoken Conversation

HW4 due