COMS 4705: Natural Language Processing
Columbia University, Fall 2026
Columbia University, Fall 2026
John Hewitt
Email: jh5020@columbia.edu
Learning from, and learning to generate, natural language is one of the core strategies in modern artificial intelligence. Systems built from the tools learned in this class are increasingly deployed in the world. This section (Section 2) provides a generative models-focused introduction to this field of natural language processing, with the goal of understanding and implementing the foundational ideas beneath state-of-the-art systems.
Topics will include: language modeling, neural network design, text tokenization, web-scale text datasets, representation learning, NLP tasks and evaluation, accelerators like GPU and TPUs, pretraining, posttraining, reinforcement learning, and many others.
For this class, it would be useful to be familiar with any of: linear algebra, python programming, probability, differential calculus. We've provided a set of notes for filling some gaps in preparation: Lecture Note 0. See here for a PDF.
Lectures: Mondays & Wednesdays, 2:40 PM – 3:55 PM
Location: Lerner Cinema
We'll be using Ed discussion forums, and Gradescope for assignment submission. You should have been added automatically to both. If you just enrolled, ping us to sync the Canvas roster.
This schedule is provisional and subject to change. Readings are optional.
| Week | Date | Lecture | Readings | Assignments |
|---|---|---|---|---|
| 1 | Mon Sep 7 | No class (Labor Day; classes begin Tue Sep 8) | ||
| Wed Sep 9 | Introduction, Language Modeling (notes) (PDF) |
A1 out a1.pdf a1.ipynb |
||
| 2 | Mon Sep 14 | Tokenization (notes) (PDF) |
||
| Wed Sep 16 | Background Review (notes) (PDF) |
|||
| 3 | Mon Sep 21 | Representation Learning 1 (Architectures) (notes) (PDF) |
||
| Wed Sep 23 | Representation Learning 2 (Learning) (notes) (PDF) |
A1 due Wednesday @ 2:30PM | ||
| 4 | Mon Sep 28 | Tasks and Evaluation (notes) (PDF) |
A2 out | |
| Wed Sep 30 | Building a Machine Translation System (notes) (PDF) |
Exam 1 Prep:
HTML ·
PDF Past exam: Exam 1 (HTML) · PDF |
||
| 5 | Mon Oct 5 | Exam 1 (on lectures through Sept 30) | ||
| Wed Oct 7 | GPUs and Parallelizable Architectures | |||
| 6 | Mon Oct 12 | Self-Attention and Transformers | ||
| Wed Oct 14 | Transformers 2 | A2 due Wednesday | ||
| 7 | Mon Oct 19 | Pretraining |
A3 Written out A3 Code out |
|
| Wed Oct 21 | Finetuning and Sampling | |||
| 8 | Mon Oct 26 | Instruction Following and RLHF | ||
| Wed Oct 28 | RLVR and Agent Alignment |
Final Project Proposal out, Oct 28 A3 Written due Fri Oct 30 |
||
| 9 | Mon Nov 2 | No class (Academic Holiday) | ||
| Wed Nov 4 | Experimental Design | |||
| 10 | Mon Nov 9 | Retrieval and Tools | A3 Code due Mon Nov 9 @ 2:40pm | |
| Wed Nov 11 | AI Safety |
Final project proposal due Wed Nov 11 @ midnight Final Project Milestone assigned, Wed Nov 11 |
||
| 11 | Mon Nov 16 | Exam 2 | A4 released, due Fri Nov 20 @ 11:59pm | |
| Wed Nov 18 | Bias, Fairness, Privacy | |||
| 12 | Mon Nov 23 | Guest Lecture TBA |
Project Milestone due November 23, 11:59PM Final Project Report assigned |
|
| Wed Nov 25 | No class (Academic Holiday / Thanksgiving) | |||
| 13 | Mon Nov 30 | History of NLP | ||
| Wed Dec 2 | Diffusion Models | |||
| 14 | Mon Dec 7 | Interpretability and Analysis | ||
| Wed Dec 9 | Looking to the Future | |||
| 15 | Mon Dec 14 | Final Project Help (last day of classes) |
||
| Wed Dec 16 | Study day | Final Project due Dec 17, 11:59PM |
This grading breakdown is provisional and subject to change.
Letter grades will be determined by the teaching staff as a function of the following breakdown; cutoffs for each letter grade will be decided at the end of the class, not by pre-set cutoffs. All written elements of the assignments, as well as the final project writeups, must be written in LaTeX and submitted as PDF.AI tools (e.g., ChatGPT, Cursor, Claude Code) are fully allowed for Assignments 1–4. While I recommend doing the assignments on your own (or with minimal AI hints) as prep for exams, you may use AI to fully solve them if you wish. It is your responsibility to ensure that submitted code and math are correct.
AI tools are also allowed for the final project, both in coding and writing. However, students must take responsibility for all written content and supporting code submitted.
No AI tools are allowed during exams, which will be written in-class.
| Names | Day | Time | Location |
|---|---|---|---|
| John | Wednesday | 5:00-6:30PM | Schermerhorn TA Area |
| John | Thursday | 10:00–11:30 AM | Schermerhorn TA Area |
Students will be evaluated and given feedback from there assigned mentor at two intermediate points in the final project process to help ensure expectations are understood. We additionally provide a document containing practical suggestions on designing and carrying out your projects in the Practical Tips For Final Projects Notes Provisional guidelines for each intermediate final project submission and brief descriptions of each are included below:
There is no attendance policy; attend as you want. though I strongly advise students to attend guest lectures, out of thanks and respect for our guest lecturers.
Please see the grading section for our policies on AI tools in this class. Otherwise, please refer to the Faculty Statement on Academic Integrity and the Columbia University Undergraduate Guide to Academic Integrity.
The teaching team is committed to accomodating students with disabilities in line with the Faculty Statement on Disability Accommodations.