COMS 4705 Lecture Notes

Natural Language Processing
Columbia University · John Hewitt

HTML + MathJax versions of the course lecture notes.

Notes

1. Lec 0: Background Mathematics · PDF
2. Lec 1: Text Representation and Language Modeling · PDF
3. Lec 2: Tokenization · PDF
4. Lec 3: Neural Networks and Sequence Representation · PDF
5. Lec 4: Neural Networks and Optimization · PDF
6. Lec 5: Tasks and Evaluations · PDF
7. Lec 6: Building a Machine Translation System · PDF
8. Lec 7: Deep Networks, GPUs and Parallelization · PDF
9. Lecs 8,9: Self-Attention and Transformers · PDF
10. Lec 11: Instruction Following and RLHF · PDF
11. Lec 12: Test-Time Scaling, Reasoning Models and Reinforcement Learning · PDF
12. Lec 13: Methods in Pretraining, Finetuning, and Generation · PDF

Course site: Course home