Pre-requisites for Language Generation and Summarization
Students must have received a B or better in COMS 4705 (NLP) or equivalent. The version of NLP that you took must have covered deep learning methods for tasks in NLP. Please provide information about your background by filling out the google form found Here . Only students who fill out the form and meet the requirements will be considered for entrance into the class. Please do note request approval by email.
|Office Hours||M 1:00-2:00, 722 CEPSR
W 4:00-5:00, 722 CEPSR
Weekly TA hours (EST) are listed below. TA hours will be held in the NLP Lab unless otherwise noted.
|Monday||Amith Ananthram (Head TA)||email@example.com||TBD|
Here is where to find these rooms on the 7th floor of CEPSR.
There has been tremendous progress recently in the development of models to generate language for different purposes – few-shot classification, document summarization, creative writing, and image captioning to name a few. This success has largely come about due to rapid advances in large language models like GPT3, PALM, Codex and Flamingo. In this class, we will explore four main topics: language generation, multimodal generation, summarization and long-format question answering. We will study large language models that have been used for these different tasks and the issues that arise with their use. For example, how can we control the output of these large language models along different dimensions? How do we evaluate the text generated by such systems? How can we develop models to produce or summarize creative texts? What are the ethical issues surrounding these kinds of models?The class will be held in person this fall. We will have some invited speakers, as shown on the syllabus. Typically an invited speaker will present for half of the class and may present remotely.
Students who take the class will have three main assignments: 1. For each class there will be a reading assignment consisting of several research papers. Students are responsible for reading all papers. 2. Students will be part of a presentation group which will be responsible for presenting a paper and raising critiques about one or more papers in class. Class participation will be graded. 3. Each student will carry out a semester-long project. This project requires submission of: a. a proposal for the project near the beginning of class; b. a midterm progress report and c. a final report and code for their project as well as a short video presentation which will be made available to the class, the Tas and the instructor for viewing.
There will be no midterm or final exam. We will use Google Cloud for the course. Instructions for setting up the cloud can be found here.
There is no text book for the class. There will be multiple research papers assigned for each class, listed next to the class in the syllabus below. Students should read the papers before class as they will need to participate in discussion of the papers.
This syllabus is still subject to change. Readings may change. But it will give you a good idea of what we will cover.
|1||Sept 7||Introduction and Course Overview||Plan then Generate|
|2||Sept 14||Large Language Models||Melanie Subbiah (Columbia)
Emily Allaway (Columbia)
|3||Sept 21||Controllable Language Generation||Tuhin Chakrabarty (Columbia)
He He (NYU)
| Metaphor Generation
|4||Sept 28|| Decoding
Diversity in Generation
MBR Decoding for NMT
|5||Oct 5||Factuality; Evaluation||Paper Presentations
Faisal Ladhak (Columbia)
Survey of Evaluation
|6||Oct 12||Generation of Creative Texts; Generation from Structured Data||Violet Peng (UCLA)
|7||Oct 19||Ethical Issues||Pamela Mishkin (Open AI)
Annotators with Attitude
|8||Oct 26||Multimodal Generation||Mohit Bansal (UNC) - Date TBD
|9||Nov 2||Summarization Introduction; Summarization Tasks||Kathy McKeown
Entity Centric Summarization
|10||Nov 9||Abstractive Summarization||Paper Presentations||
|11||Nov 16||Summarization of Different Genres
|Miguel Ballesteros (Amazon)
|12||Nov 30||Factuality; Medical Summarization||Noemie Elhadad (DBMI, Columbia)
|Hospital Course Summarization
|13||Dec 7||Query-focused Summarization; Long Format Question Answering||Paper Presentations||
Problems with ELI-5
Check EdStem for announcements and discussion. Check courseworks for your grades (only you will see them).. All questions should be posted through EdStem instead of emailing Professor McKeown or the TAs. They will monitor the discussion lists to answer questions.
Copying or paraphrasing someone's work (code included), or permitting your own work to be copied or paraphrased, even if only in part, is not allowed, and will result in an automatic grade of 0 for the entire assignment or exam in which the copying or paraphrasing was done. Your grade should reflect your own work. If you believe you are going to have trouble completing an assignment, please talk to the instructor or TA in advance of the due date.