MACHINE LEARNING September 1, 2014
COMS4771001
COURSE INFO
Time
& Location 
T/Th
1:10pm2:25pm at 301 Pupin 
Instructor 
Professor
Tony Jebara, jebara(at)cs(dot)columbia(dot)edu 
Office
Hours 
T/Th 2:30pm3:15pm at 605 CEPSR 
TAs 
Ayushi Singhal, as4521(at)columbia(dot)edu Wenting Yin, wy2214(at)columbia(dot)edu Ruichi Yu, ry2254(at)columbia(dot)edu Sarper Sertoglu, ss4552(at)columbia(dot)edu Michael Yang, my2292(at)columbia(dot)edu

Bulletin
Board 
Available via courseworks.columbia.edu and is the best

Prerequisites: Knowledge of linear algebra
and introductory probability or statistics.
Description: This course introduces topics in machine learning for both generative
and discriminative estimation. Material will include least squares methods, Gaussian
distributions, linear classification, linear regression, maximum likelihood, exponential
family distributions, Bayesian networks, Bayesian inference, mixture models, the EM
algorithm, graphical models, hidden Markov models, support vector machines, and
kernel methods. Students are expected to implement several algorithms in Matlab
and have some background in linear algebra and statistics.
Required
Texts:
Michael
I. Jordan and Christopher M. Bishop, Introduction to Graphical Models.
Still
unpublished. Available online via courseworks.columbia.edu
Christopher M. Bishop, Pattern Recognition
and Machine Learning, Springer.
2006
First Edition is preferred. ISBN: 0387310738. 2006.
Optional
Texts: Available
at library (additional handouts will also be given).
Tony Jebara, Machine Learning: Discriminative and Generative, Kluwer, 2004
ISBN: 1402076479. Boston, MA, 2004.
R.O. Duda, P.E. Hart and D.G. Stork, Pattern
Classification, John Wiley & Sons, 2001.
Trevor
Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical
Learning.
SpringerVerlag New York USA, 2009. 2nd Edition. ISBN 0387848576.
Graded
Work: Grades will
be based on 5 homeworks (45%), the midterm (20%),
two surprise inclass quizzes (5%), and
the final exam (30%). Any material covered in
assigned readings,
handouts, homeworks, solutions, or lectures may appear in exams.
If you miss the midterm and don't have an official reason, you will get 0 on it.
If you have an official reason, your midterm grade will be based on the final exam.
If you miss a quizz and don't have an official reason, you will get 0 on it.
If you have an official reason, your missed quiz grade will be based on the final exam.
Tentative
Schedule:
Date 
Topic 
September 2 
Lecture 01: Introduction 
September 4 
Lecture 02: Least Squares 
September 9 
Lecture 03: Linear Classification and Regression 
September 11 
Lecture 04: Neural Networks and BackProp 
September 16 
Lecture 05: Neural Networks and BackProp 
September 18 
Lecture 06: Support Vector Machines 
September 23 
Lecture 07: Support Vector Machines 
September 25 
Lecture 08: Kernels and Mappings 
September 30 
Lecture 09: Probability Models 
October 2 
Lecture 10: Probability Models 
October 7 
Lecture 11: Bernoulli Models and Naive Bayes 
October 9 
Lecture 12: Multinomial Models for Text 
October 14 
Lecture 13: Graphical Models Preview 
October 16 
Lecture 14: Gaussian Models 
October 21 
Lecture 15: Gaussian Regression and PCA 
October 23 
MIDTERM 
October 28 
Lecture 16: Bayesian Inference 
October 30 
Lecture 17: The Exponential Family 
November 4 
ELECTION DAY (NO CLASS) 
November 6 
Lecture 18: Mixture Models and Kmeans Clustering 
November 11 
Lecture 19: Expectation Maximization 
November 13 
Lecture 20: Expectation Maximization 
November 18 
Lecture 21: Graphical Models 
November 20 
Lecture 22: Graphical Models 
November 25 
Lecture 23: Junction Tree Algorithm 
November 27 
THANKSGIVING DAY (NO CLASS) 
December 2 
Lecture 24: Junction Tree Algorithm 
December 4 
Lecture 25: Hidden Markov Models 
December ?? 
COMPREHENSIVE FINAL EXAM 
Class
Attendance: You
are responsible for all material presented in the class
lectures,
recitations, and so forth. Some material will diverge from the textbooks
so
regular attendance is important.
Late
Policy: If you
hand in late work without approval of the instructor or TAs,
you will
receive zero credit. Deadlines are nonnegotiable.
Cooperation
on Homework:
Collaboration on solutions, sharing or copying of
solutions
is not allowed. Of course, no cooperation is allowed during exams.
This
policy will be strictly enforced.
Web
Page: The class
URL is: http://www.cs.columbia.edu/~jebara/4771
and
will
contain copies of class notes, news updates and other information.
Matlab: We'll use Matlab for coding, download it at www.cs.columbia.edu
by clicking on: > Computing > Software > Matlab.
Note: use JDK 1.6 instead of JDK 1.7.