COMS E6998-3: Machine Learning for Natural Language Processing (Spring 2011)




Problem sets


Date Lecture Notes etc
Wed. January 19th Lecture 1: Introduction, hidden Markov models, log-linear models Slides part 1 (introduction, hidden Markov models), Slides part 2 (log-linear models).

Background reading: J&M Chapter 5 on part-of-speech tagging, J&M sections 6.1-6.4 (HMMs), J&M 6.6-6.7 (Log-linear models). Note that there are a couple of typos in section 6.7 in log-linear models, described here.

Wed. January 26th Lecture 2: Log-linear models, MEMMs, CRFs Slides (log-linear models, MEMMs, CRFs)

Required reading: note on log-linear models, MEMMs, and CRFs

Wed. February 2nd Lecture 3: Discriminative Dependency Parsing Slides (Discriminative dependency parsing)


McDonald, Crammer and Pereira, 2005 (on projective discriminative dependency parsing)

McDonald, Pereira, Ribarov, Hajic, 2005 (on non-projective discriminative dependency parsing, including a description of the decoding algorithm for directed spanning trees)

Koo, Globerson, Carreras, Collins, 2007 (on calculating marginals using the matrix-tree theorem).

Wed. February 9th Lecture 4: CRFs for CFG parsing, the structured perceptron, belief propagation Slides
Wed. February 16th Lecture 5: Dual Decomposition (part I) Slides (4 per page), Slides (1 per page), with animations

Reading: Koo et al., EMNLP 2010

Wed. February 23rd Lecture 6: Dual Decomposition (part II) Slides (part 1), Slides (part 2)

Reading: Rush et al., EMNLP 2010

Wed. March 2nd Lecture 7 Student project proposal presentations.
Wed. March 9th Lecture 8: Statistical Machine Translation Slides on phrase-based models(4 per page), Slides on Hiero, Koehn and Knight tutorial.

Readings: Chiang, 2007, Koehn et al., 2003.

Wed. March 16th Lecture 9:
Wed. March 30th Lecture 10: Lagrangian relaxation for MT decoding Slides
Wed. April 6th Lecture 11: Discriminative models for MT/Brown clustering and semi-supervised learning Slides part 1, Slides part 2
Wed. April 13th Lecture 12: Yarowsky's algorithm, cotraining Slides part 1, Slides part 2
Wed. April 20th Lecture 13:
Wed. April 27th Lecture 14: