COMS E6998-1: Advanced Machine Learning


This graduate course covers current research level topics in machine learning for both
generative and discriminative estimation. Material will include exponential family distributions,
Bayesian networks, Bayesian inference, maximum likelihood, maximum entropy, mixture
models, the EM algorithm, graphical models, hidden Markov models, variational
methods, linear classifiers, regression, generalization bounds, support vector machines,
kernel methods and transduction.


Projects Powerpoint Files

Readings Schedule

Readings on Variational Methods:
Jordan's Intro (ps.gz)
Jaakkola's Tutorial on Mean Field (ps)
Jaakkola on QMR-DT (pdf)

Readings on Support Vector Machines:
Chris Burges' Tutorial (ps.gz)

Assignment 1: as Postscript or as PDF
Assignment 2: as Postscript or as PDF and data: Dataset1 Dataset2 Dataset3 Dataset4
Assignment 3: as Postscript or as PDF and data: Dataset5 Dataset6 Dataset7 and code: sampleNet likeNet learnNet
Assignment 4: as Postscript or as PDF
Class Project: as Postscript or as PDF

Matlab Tutorial and Useful Functions

Scanned Class Notes (pdf) pages: (1-5) (6-8) (9-16) (17-25) (26-35) (36-40) (41-46) (47-54)


WHAT'S NEW

COURSE BENEFITS

  • Formal treatment of machine learning with statistics and graphical models
  • Classification, modeling and prediction for many applied domains
  • Research level exploration of current issues and trends in the field
PROFESSOR JEBARA
  • Assistant Professor of Computer Science
  • Various publications and research in machine learning community
  • Applications in human-computer interfaces, computer vision
Lecturer/Manager: Tony Jebara
Office Hours: CEPSR 605, Tuesdays 2pm-4pm and Thursdays 2:30pm-3:30pm
Office Phone: 212-939-7079
E-mail Address: jebaraATcsDOTcolumbiaDOTedu

Day & Time of Class: Mondays 16:10-18:00
Class Location: 1024 MUDD
Class Homepage: http://www.cs.columbia.edu/~jebara/6998-01
Credits for Course: 3
Class Type: Lecture

Prerequisites: Linear Algebra, Introductory Machine Learning or Introductory Statistics
Required Text(s):
  • Introduction to Graphical Models by M. Jordan and C. Bishop. The authors have agreed to let us use the current online draft of the text which is scheduled to be published shortly. The files will be made available through a secure web page however, they have asked that these do not circulate outside the course. Please respect this while you use the online version.

    You will need to send me an email with a password (make up a NEW password just for this class) to see the book (as postscript or pdf files). A couple of days after you have mailed me, you should be able to follow this link: http://www.cs.columbia.edu/~jebara/6998-01/book . I will setup your user name from the first part of your email address which you send me the email with (i.e. 'joe@columbia.edu' will have a username 'joe'). Include a new made up password in the body of your email which will be attributed to your username. Please use '6998' as the title of your email.
    • Reference Text(s):
  • Pattern Classification by Duda, Hart and Stork
  • Neural Networks for Pattern Recognition by Chris Bishop
  • Papers and handouts will be made available later in the term
    		•
    Homework(s): Roughly 5 problem sets. These will be assigned and due every 2 weeks.
    Project(s): A research project is required that uses course material in an applied setting or develops it further
    Paper(s): A conference style paper describing the project will be due at the end of the term.
    Grading: Problem Sets 50% and Project (paper & presentation) 50%
    Software Requirements: Programming (Matlab or C)
    Homework Submission: Due in class or email by start of class

    COURSE OUTLINE FOR CURRENT SEMESTER

    Tentative schedule (subject to change)
    Course Outline for COMS E6998-1: Advanced Machine Learning
    Original
    Date
    Class
    No.
    Semester
    Date
    Topics/Chapters Covered
    Assigned
       Due   
    Jan. 28  1    Distributions, Bayesian Inference     
    Feb. 4  2    Exponential Family and ML     
    Feb. 11  3    Mixture Models and the EM Algorithm     
    Feb. 18  4    Generative and Discriminative Learning     
    Feb. 25  5    Graphical Models     
    Mar. 4  6    Junction Tree Algorithm     
    Mar. 11  7    Hidden Markov Models     
    Mar. 25  8    Approximate and Variational Methods     
    Apr. 1  9    Loopy Propagation     
    Apr. 8  10    Generalization and Model Selection     
    Apr. 15  11    Support Vector Machines, Kernels     
    Apr. 22  12    Transduction, Feature Selection     
    Apr. 29  13    Maximum Entropy, Duality     
    May 6  14    Project Presentations