COMS 6998: Advanced Topics in Spoken Language Processing

Instructors: Julia Hirschberg and Sarah Ita Levitan

Time: F 10:10-12:00 (Spring 2019)

Location: Mudd 627

 

Prerequisite: COMS 4705 or another speech or NLP class

Description:  This class will introduce students to spoken language processing:  basic concepts, analysis approaches, and applications.

Required readings:

Jurafsky & Martin 2019 chapters

These and other readings are linked from this syllabus for each class.

Suggested:

Keith Johnson. Acoustic & Auditory Phonetics (3rd edition). Wiley.  2011.

 

Resources:

A list of resources can be found here.

 

Office Hours

Julia Hirschberg: Friday, 12-1 (CEPSR 705)

Sarah Ita Levitan: Thursday, 11-12 (CEPSR 701)

Rose Sloan: Wednesday, 3-4 (CEPSR 7LW3 -- Speech Lab)

Zixiaofan (Brenda) Yang: Tuesday, 2-3 (CEPSR 7LW3 -- Speech Lab)

Grade Breakdown

5% attendance and participation

15% weekly posts

15% HW1

15% HW2

25% HW3

25% HW4

 

 

Academic Integrity

The SEAS academic integrity policy is found here.

The CS academic integrity policy is found here.

Syllabus

Note: Schedule and readings are subject to change

 

Date	Topic	Readings	Assignments
Week 1: 1/25	Introduction to Speech Processing
Week 2: 2/1	From Sounds to Language	Jurafsky & Martin Chapter 7 (sections 1-3)
Week 3: 2/8	Acoustics of Speech	Jurafsky & Martin Chapter 7 (sections 4-7)
Week 4: 2/15	Tools for Speech Analysis	Praat Tutorial(Chapter 11 - scripting - is optional) Download Praat	HW1: Praat Recording and Analysis (assigned)
Week 5: 2/22	Analyzing Speech Prosody	ToBI Conventions Modeling Prosody Prosody and Meaning	HW1 due
Week 6: 3/1	Text-to-Speech Synthesis	Jurafsky & Martin Chapter 8 Merlin Tutorial	HW2: ToBI Labeling (assigned)
Week 7: 3/8	Speech Recognition: Then and Now	Jurafsky & Martin Chapter 9 Deng & Yu Chapter 7
Week 8: 3/15	Speech Analysis: Entrainment in Spoken Language Guest lecture: Rivka Levitan Guest lecture: Zhihua (Shirley) Xia Entrainment in Deceptive Speech	Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions Mark My Words! Linguistic Style Accommodation in Social Media Prosodic entrainment in Mandarin and English: a cross-linguistic comparison	HW2 due
Week 10: 3/29	Spoken Dialogue Systems	Jurafsky & Martin Chapter 24 Jurafsky & Martin Chapter 25	HW3: Dialogue Acts (assigned)
Week 11: 4/5	Speech Analysis: Personality and Mental State	Detecting late-life depression in Alzheimer's disease through analysis of speech and language Vocal-Source Biomarkers for Depresion: A Link to Psychomotor Activity Automatic Recognition of Personality in Conversation
Week 12: 4/12	Speech Analysis: Deception and Trust	Linguistic Cues to Deception and Perceived Deception in Interview Dialogues Lying Words: Predicting Deception from Linguistic Styles Personality Factors in Human Deception Detection: Comparing Human to Machine Performance	HW3 due
Week 13: 4/19	Speech Analysis: Emotion, Sentiment and Keyword Search	Classifying Subject Ratings of Emotional Speech Using Acoustic Features Using Context to Improve Emotion Detection in Spoken Dialog Systems Adieu features? end-to-end speech emotion recognition using a deep convolutional recurrent network	HW4: Emotional Speech Detection (assigned)
Week 14: 4/26	Speech Analysis: Sarcasm and Humor	Sarcastic or Not: Word Embeddings to Predict the Literal or Sarcastic Meaning of Words "Sure, I did the right thing": A system for sarcasm detection in speech "Yeah, right": Sarcasm recognition for spoken dialogue systems
Week 15: 5/3	Speech Analysis: Charisma, Likability and Style	Charisma perception from text and speech "Would You Buy A Car From Me?"-- On the Likability of Telephone Voices Extracting Social Meaning: Identifying Interactional Style in Spoken Conversation	HW4 due