CS 6998 Section 011: Language Generation and Summarization, Fall 2023

Pre-requisites for Language Generation and Summarization

Students must have received a B or better in COMS 4705 (NLP) or equivalent. The version of NLP that you took must have covered deep learning methods for tasks in NLP. Please provide information about your background by filling out the google form found Here. The google form also includes an assignment that must be completedin order to be eligible. The assignment is due by the day after the first class (Thursday, September 7th), but we will be accepting a percentage of the class early for forms completed before August 26th. Only students who fill out the form and meet the requirements will be considered for entrance into the class. Please do note request approval by email.

Course Information

Time W 4:10-6:00pm
Place TBD
Professor Kathleen McKeown
Office Hours M 4:00-5:00, 722 CEPSR
W 6:00-7:00, 722 CEPSR
Email kathy@cs.columbia.edu
Phone 212-939-7118

Weekly TA hours (EST) are listed below. TA hours will be held in the NLP Lab unless otherwise noted.

Thursday Melanie Subbiah m.subbiah@columbia.edu Th 1:15-3:15, 724 CEPSR
Tuesday Nick Deas ndeas@cs.columbia.edu T 2:00-4:00, 7LW1 CEPSR

Here is where to find these rooms on the 7th floor of CEPSR.

Course Description

There has been tremendous progress recently in the development of models to generate language for different purposes – few-shot classification, document summarization, creative writing, and image captioning to name a few. This success has largely come about due to rapid advances in large language models like GPT3, ChatGPT, GPT4, PALM, T5, Alpaca and Flamingo. In this class, we will explore four main topics: language generation, multimodal generation, summarization and long-format question answering. We will study large language models that have been used for these different tasks and the issues that arise with their use. For example, how can we control the output of these large language models along different dimensions? How do we evaluate the text generated by such systems? How can we develop models to produce or summarize creative texts? What are the ethical issues surrounding these kinds of models?

The class will be held in person this fall. We will have some invited speakers, as shown on the syllabus. Typically an invited speaker will present for half of the class and may present remotely.


Students who take the class will have three main assignments: 1. For each class there will be a reading assignment consisting of several research papers. Students are responsible for reading all papers. 2. Students will be part of a presentation group which will be responsible for presenting a paper and raising critiques about one or more papers in class. Following presentation, smaller discussion groups will be formed to discuss each paper. Class participation will be graded. 3. Each student will carry out a semester-long project. This project requires submission of: a. a proposal for the project near the beginning of class; b. a midterm progress report and c. a final report and code for their project as well as a short video presentation which will be made available to the class, the TAs and the instructor for viewing.

Late submission policy: You have 4 free late days to use across the 3 assignments for the course (proposal, midterm, final). Once you have used all your days, then you will lose 7% of the points per day late unless you have demonstrated a valid medical excuse.

There will be no midterm or final exam. We will use Google Cloud for the course. Instructions for setting up the cloud can be found here.


There is no text book for the class. There will be multiple research papers assigned for each class, listed next to the class in the syllabus below. Students should read the papers before class as they will need to participate in discussion of the papers.


This syllabus is still subject to change. Readings may change. But it will give you a good idea of what we will cover.

Week Class Topic Speaker Reading Assignments
1 Sept 6 Introduction and Course Overview Plan then Generate (Sept 7) Enrollment Form
2 Sept 13 Large Language Models Melanie Subbiah (Columbia)
Kathy McKeown
Instruction Tuning
Alpaca Farm
Resources for open source models
3 Sept 20 Controllable Language Generation Kathy McKeown
Emily Allaway (Columbia)
Reward Gaming in Conditional Text Generation
A Recipe For Arbitrary Text Style Transfer with Large Language Models
Truncation Sampling as Language Model Desmoothing

4 Sept 27 Creative Language Generation Tuhin Charkrabarty (Columbia)
Paper Presentations

ByGPT5: End-to-End Style-conditioned Poetry Generation
Zero-Shot Sonnet Generataion
Creative Writing with an AI-Powered Writing Assistant
5 Oct 4 Explanation Generation and Evaluation Chenhao Tan (U Chicago)
Paper Presentations
Towards a Science of Human-AI Decision Making

Do Models Explain Themselves?
SCOTT: Self-Consistent Chain-of-Thought
Unsupervised Selective Rationalization
Semester Project Proposal
6 Oct 11 Multimodal Generation Amith Ananthram (Columbia))
Paper presentations

Hierarchical3D Adapters
Ambiguity with Images
Scaling Laws
7 Oct 18 Deployment and Ethics Pamela Mishkin (OpenAI)
Paper Presentations

Trails of Political Biases
Annotators with Attitude
8 Oct 25 Bias Nick Deas (Columbia)
Paper Presentations

African American Language Bias

Adding Instructions during Pretraining
COBRA Frames
Memorization of Conspiracy Theories
9 Nov 1 Summarization Introduction and Tasks Kathy McKeown
Paper Presentations

Conversation Benchmark
Neutral multi-news
10 Nov 8 Controllable Summarization and Diffusion Jacob Andreas (MIT)
Paper Presentations

Comparative Opinion Summarization
Controllable Diffusion
Midterm Project Update
11 Nov 15 Summarization Evaluation Alex Fabbri (SalesForce)
Paper Presentations

Multi-Dimensional Summarization Eval
Factual Errors
12 Nov 29 Emotion Summarization and Multilingual Summarization Jessy Li (UT Austin)
Paper Presentations
Emotion Triggers

Multilingual Summarization
Multilingual Not Multicultural
Code-Switch Synthesis
13 Dec 6 Summarization Beyond News Kathy McKeown

Paper Presentations
Calibration Sets
(Dec. 13) Final Paper


Check EdStem for announcements and discussion. Check courseworks for your grades (only you will see them).. All questions should be posted through EdStem instead of emailing Professor McKeown or the TAs. They will monitor the discussion lists to answer questions.

Academic Integrity

Copying or paraphrasing someone's work (code included), or permitting your own work to be copied or paraphrased, even if only in part, is not allowed, and will result in an automatic grade of 0 for the entire assignment or exam in which the copying or paraphrasing was done. Your grade should reflect your own work. If you believe you are going to have trouble completing an assignment, please talk to the instructor or TA in advance of the due date.