COMS W3261: Computer Science Theory, Fall 2023

Announcements

General information

The two sections share the same gradescope and edstem pages (both linkable through courseworks).

Teaching staff and Office Hours

  • Professor: Toniann Pitassi
  • Head TAs: Alice Chen (yc3877@columbia.edu), Helen Chu (hc2932@columbia.edu)
  • TAs: Angel Cui (lc3542@barnard.edu), Ziheng Huang (zh2556@columbia.edu), William Pires (wp2294@columbia.edu), Sayak Chakrabarti (sc5084@columbia.edu), Jeannie Ren (jr3766@columbia.edu), Yihan Shen (ys3524@columbia.edu), Matthew Weng (zw2815@columbia.edu), Nicholas Zhang (nz2368@columbia.edu), Daniel Wang Cheng (dwc2119@columbia.edu)
If you have any questions about the course, please send your email to: cs3261admin@googlegroups.com. This email will be seen by the Professor as well as the head TAs. Questions with sensitive personal information can be emailed to just the Professor.

Office hours: The full schedule can be found in the course calendar:

Please check on the location for each of the office hours, e.g., zoom or a specific room. Also before going to office hours, check the calendar for any last minute changes.

Class description and syllabus

This course is an introduction to models of computation, computability, and complexity. We will ask questions like: How do we define the abstract notion of "computation" or a "computer"? What problems can be computed? What problems can be computed efficiently? Can we prove that some problems can never be computed, no matter how fast our computers become? The course is highly mathematical and abstract; in particular, there will be minimal programming assignments (if any) and homework problems will frequently involve proving or disproving various assertions. Students are expected to be comfortable with the material covered in Discrete Math (W3203) or the equivalent.

Topics (not exhaustive): automata and the languages they define, context-free grammars and the languages they define, Turing machines, basic complexity theory (such as P vs. NP).

Below is a tentative outline of material covered per lecture. This will be updated throughout the course.
  • (9/6) Lecture 1: Lecture notes Introduction to class, background definitions. Deterministic Finite Automata.
    Readings: Chapter 0, Chapter 1.1
  • (9/11) Lecture 2: Lecture notes DFAs, NFAs
    Readings: Chapter 1.2
  • (9/13) Lecture 3: Lecture notes NFAs, Equivalence of DFA and NFAs, Closure Properties
    Readings: Chapter 1.2
  • (9/18) Lecture 4 (HW1 OUT): Lecture notes Regular Expressions, Equivalence with Regular Languages
    Readings: Chapter 1.3
  • (9/20) Lecture 5: Lecture notes Equivalence of Regular Expressions and Regular Languages, Start of Pumping Lemma
    Readings: Chapter 1.3 and Handouts below on NFA to Regular Expression Conversion
  • (9/25) Lecture 6: Lecture Notes Pumping Lemma
    Readings: Chapter 1.4
  • (9/27) Lecture 7: Lecture Notes Wrapup Finite Automata
  • (10/2) Lecture 8 (HW1 DUE, HW2 OUT): Lecture Notes Start of Context Free Languages: PDAs and CFLs
    Readings: Chapter 2.2
  • (10/4) Lecture 9: Lecture Notes PDAs and CFLs
    Readings: Chapter 2.2, 2.1
  • (10/9) Lecture 10: Lecture Notes CLFs and Pumping Lemma for CFLs
    Readings: Chapter 2.3
  • (10/11) Lecture 11: Lecture Notes Problems on CFLs
  • (10/16) Lecture 12 (HW2 DUE): Lecture Notes Equivalence between PDAs and CFLs
    Readings: Chapter 2.4
  • (10/18) Lecture 13 Lecture Notes Midterm Review
  • (10/23) TEST 1
  • (10/25) Lecture 14 Lecture Notes Turing Machines
    Readings: Chapter 3.1
  • (10/30) Lecture 15 Lecture Notes Turing Machines cont'd
    Readings: Chapter 3
  • (11/1) Lecture 16 Lecture Notes Decidability and Diagonalization
    Readings: Chapter 4
  • (11/6) Academic Holiday (No Class, HW3 OUT)
  • (11/8) Lecture 17 Lecture Notes Decidability and Reducibilities
    Readings: Chapters 4,5
  • (11/13) Lecture 18 Lecture Notes Reducibilities cont'd
    Readings: Chapter 5
  • (11/15) Lecture 19 Lecture Notes Reducibilities and Practice Problems
    Readings: Chapter 5
  • (11/20) Lecture 20 Lecture Notes Intro to Complexity Theory, the class P
    Readings: Chapter 7.1,7.2
  • (11/21) HW3 Due, HW4 OUT
  • (11/22) Academic Holiday (No class)
  • (11/27) Lecture 21 Lecture Notes NP and NP-completeness
    Readings: Chapter 7.3,7.4,7.5
  • (11/29) Lecture 22 Lecture Notes NP-completeness cont'd
    Readings: 7.3-7.5
  • (12/4) Lecture 23 Lecture Notes NP-completeness
  • (12/5) (HW4 DUE)
  • (12/6) Lecture 24 Lecture Notes See Handouts for the accompanying review problems for Test 2
  • (12/11) TEST 2 (Last Class)

Homework

Homework should be submitted via Gradescope. Anonymous homework grading is enabled, so please avoid writing your name or other identifying information on the pages you link to the different problems to be graded on gradescope. If you had discussion collaborators, include their name on a separate page of your submission, not linked to any problem. We recommend that you typeset your homework (e.g. using LaTeX), but you may also submit scanned legibly handwritten homework. We will not spend much effort trying to decipher handwriting we find illegible.

Lateness policy: Late homework will not be accepted. However for emergencies, such as a serious family or medial emergency, assuming you can provide all necessary documentation, we will make every effort to provde accomodations.

Note: The responsibility for what is written on a submitted homework belongs solely to the student who submitted it. The teaching staff will try their best to help students and answer their questions. A primary goal of this class is for you to be able to tell correct solutions from wrong ones. We aim (through lectures, office hours, and homeworks) to to help you understand the material, to a point where you understand what exactly the question is asking, and how to tell for yourself whether you had succeeded to solve it. In particular, if a TA (or the professor) tell a student something, even if they accidentally make a mistake in their reasoning, the student is still responsible for catching that (at least if they are going to write something based on it), by fully understanding the problem and solution.

Grading and Class Policies

Grading policy

The final grade is determined by homework (28%), and two exams (72%).

We will accept regrade requests for assignments on Gradescope up to one week after grades are released. Please read the published solutions and the rubric before submitting a regrade request. To submit a regrade request on Gradescope, click on the question and click "Request Regrade", then write a few sentences explaining why you think your answer is correct. When we receive a regrade request, if we realize that we made a mistake in applying the rubric, we will correct the mistake. Note that this may result in the grade going down (or up, or no change).

There will be two exams, in class:

  • Exam 1: in class, Monday October 23th.
  • Exam 2: in class, Monday December 11th (last class).

Collaboration policy

All students are assumed to be aware of the computer science department academic honesty policy .

For homework, collaboration in groups of two or three students is allowed (but not required). If you do collaborate, you must write your own solution (without looking at anybody else's solutions), and list the names of anyone with whom you have discussed the problems. If you use a source other than the textbook in doing your assignment, explain the material from the source in your own words and acknowledge the source. In any case, you are not allowed to look at written solutions or written notes from any other student, even if you worked on solving the homework together. Collaboration without acknowledgment is considered cheating. Obviously, you are also not allowed to look at solutions from other classes, other institutions, previous years, and so on. If you are in doubt, ask the professor. Homework should constitute your own work.

Every student caught cheating will be referred to Columbia's office of Student Conduct and Community Standards, as well as subject to academic penalty in the class.

Textbook

Readings and homeworks will be assigned from Introduction to the Theory of Computation by Michael Sipser. Either the Second or Third Edition is acceptable. The book's website and errata are available here. We will generally follow the book in class, but some departures are possible. Students are responsible for all material taught in class.

Optional text: Introduction to Automata Theory, Languages and Computation by John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman

Do I have the Prerequisites for the Class?

The course requires Discrete Math (COMS W3203) as a prerequisite. To check and refresh on your discrete math prerequisite for the class, read Chapter 0 of Sipser's textbook (here is a pdf of this chapter if you don't have the book yet). Quiz 1 will also test some discrete math background. You can also check Tim Randolph's HW0 (here are the solutions). Finally, check out handout 1 below, with a discrete math review sheet. If you have difficulty with any of the above, and have not yet taken 3203 discrete math, you should drop this class and take 3203 first.

Handouts and Review Resources

Below are handouts generated by the teaching staff as a supplement to class material. These do not constitute required material for the class, but rather meant to support your learning and understanding. Most of the resources below provide further examples and review of what was already covered in class, but some of the resources provide new (and completely optional) more advanced material for interested students. We will also have occasional (and optional) in-person review sessions -- those will also be accompanied by handouts that will be posted below for all students.

Useful and/or Interesting links

  • Tools for drawing finite automata, LaTex, etc:
    • TikZ is a TeX package for programmatically creating graphics which can be used to draw automata. See an example here or here.
    • Finite State Machine Designer is a useful tool to draw automata, which can then be exported to LaTeX (using the TikZ package above).
    • Java Formal Languages and Automata Package (JFLAP) is a useful Java tool that allows you to build automata, test out their functionality, and easily modify them.
    • GasTeX is a LaTeX plugin for easily drawing finite automata and graphs. See also JasTex, a small Java applet for visually drawing finite automata. It automatically generates GasTeX code.
    • Detexify lets you search for LaTeX symbols by drawing them.
    • Other useful LaTeX packages (included with the standard Texlive distribution) include amsmath, amsthm and amssymb (for typesetting equations), and xyfig (for typesetting automata and trees).
  • Turing's paper "On Computable Numbers, with an Application to the Entscheidungsproblem", published 1936. (possibly more readable font for the symbols here).
  • If you want to see an explicit specific example of how you might encode a TM as a string, you can check section 2 in this document (written by TA Freddy Kellison-Linn who covered a lecture in 2018).
  • Some short videos suggested by students in the class to help understand proofs by diagonalization:
    • Numberphile short video on countability and on uncountability of the reals (Cantor's proof).
    • video on the undecidability of the halting problem (via a diagonalization proof).
  • Scooping the loop snooper: A proof that the Halting problem is undecidable, in the style of Dr. Seuss, written by linguist Geoffrey Pullum to honor Alan Turing. Copied from his page .
  • Who Can Name the Bigger Number?: An essay by Scott Aaronson, related to uncomputability and Turing Machines (read about the Busy Beaver problem, and more).