Project

Overview

The course project is an opportunity to engage in/with research on machine learning theory. You are free to pick any topic you like, within reason.

You may work individually or in pairs. The expectation for projects done in pairs is naturally higher than for those done individually.

Examples

Some examples of suitable project “types” are as follows.

  • Reading project: Read a few research papers (2-3 papers, depending on how substantial they are) on a topic in machine learning theory (broadly construed), and write a short survey that unifies and clarifies the main results, ideas, techniques, etc.
  • Research project: Prove a new and interesting theoretical result in a new or existing model. You can be ambitious, but do also aim for something interesting to show by the end of the semester. For example, a perfectly reasonable project is to find a simpler proof of an existing but complicated result. You can also “convert” a research project to a reading project.
  • Implementation project: Develop a practical implementation of a non-trivial learning algorithm from the machine learning theory literature, and conduct a careful empirical evaluation of it in a real-world domain.

These are only suggestions; you are welcome to come up with your own project type provided that you discuss it with (and have it approved by) me first.

Instructions and deadlines

  • Prior to Friday, March 13: Come to office hours or schedule a meeting with me to discuss your plan for the project.
  • Friday, March 13: Submit a project proposal (a page or two should suffice), describing the project type, the chosen topic, and evidence of some preliminary “leg-work” (e.g., a clear problem statement, some well-informed conjectures, some milestones with self-imposed deadlines, relevant bibliographic references).
    • I’ll provide feedback if necessary, and you may need to go through a few iterations of the proposal before it is finally approved.
  • Monday, April 6: Submit a progress report (a page or two should suffice) that explicitly refers back to your project proposal: what has been accomplished, what goals should be revised, etc.?
  • Monday, May 4: Submit a final project report. Try to limit its length to at most eight pages; quality and clarity are more important than quantity. Please just email the report to the instructor.

Each of the project proposal, progress report, and final project report should be neatly typeset as a PDF document using TeX, LaTeX, or similar systems with bibliographic references (e.g., using BibTeX).

We would like to have a day or two of oral project presentations at the end of the semester (especially for those who work in pairs). This might come either in the last day of classes or during the final exam period. Update 3/15: This is unlikely to happen; so please just plan to submit a project report by May 4.

Where to find papers on learning theory

One place to find papers on learning theory is the proceedings of the Conference on Learning Theory (COLT; formerly known as the Conference on Computational Learning Theory).

Another conference, called Algorithmic Learning Theory (ALT), also exclusively focuses on learning theory.

The Conference on Neural Information Processing Systems (now known as NeurIPS) and the International Conference om Machine Learning (ICML) are general machine learning conferences that also feature many papers on theory.

Other good conferences on machine learning include the International Conference on Artificial Intelligence and Statistics and the Conference on Uncertainty in Artificial Intelligence.

Occasionally, some journals (like JMLR, JCSS, Algorithmica) have special issues on learning theory. A good way to find this is to search “special issue” and “learning theory” on your favorite search engine. (Perhaps also throw in the name of the journal in the query.) These same journals are also likely to have regular journal articles on learning theory (not as part of a special issue).