MACHINE LEARNING September 8, 2015
COMS4771001
COURSE INFO
Time
& Location 
T/Th
10:10am11:25am at 501 NWC 
Instructor 
Professor
Tony Jebara, jebara(at)cs(dot)columbia(dot)edu 
Office
Hours 
W 2:00pm4:00pm at 605 CEPSR 
TAs 
Jialu Zhong, jz2612(at)columbia(dot)edu Robert Ying, ry2242(at)columbia(dot)edu

Bulletin
Board 
Available via courseworks.columbia.edu and is the best

Prerequisites: Knowledge of linear algebra
and introductory probability or statistics.
Description: This course introduces topics in machine learning for both generative
and discriminative estimation. Material will include least squares methods, Gaussian
distributions, linear classification, linear regression, maximum likelihood, exponential
family distributions, Bayesian networks, Bayesian inference, mixture models, the EM
algorithm, graphical models, hidden Markov models, support vector machines, and
kernel methods. Students are expected to implement several algorithms in Matlab
and have some background in linear algebra and statistics.
Required
Texts:
Michael
I. Jordan and Christopher M. Bishop, Introduction to Graphical Models.
Still
unpublished. Available online via courseworks.columbia.edu
Christopher M. Bishop, Pattern Recognition
and Machine Learning, Springer.
2006
First Edition is preferred. ISBN: 0387310738. 2006.
Optional
Texts: Available
at library (additional handouts will also be given).
Tony Jebara, Machine Learning: Discriminative and Generative, Kluwer, 2004
ISBN: 1402076479. Boston, MA, 2004.
R.O. Duda, P.E. Hart and D.G. Stork, Pattern
Classification, John Wiley & Sons, 2001.
Trevor
Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical
Learning.
SpringerVerlag New York USA, 2009. 2nd Edition. ISBN 0387848576.
Graded
Work: Grades will
be based on 5 homeworks (45%), the midterm (20%),
two surprise inclass quizzes (5%), and
the final exam (30%). Any material covered in
assigned readings,
handouts, homeworks, solutions, or lectures may appear in exams.
If you miss the midterm and don't have an official reason, you will get 0 on it.
If you have an official reason, your midterm grade will be based on the final exam.
If you miss a quizz and don't have an official reason, you will get 0 on it.
If you have an official reason, your missed quiz grade will be based on the final exam.
Tentative
Schedule:
Date 
Topic 
September 8 
Lecture 01: Introduction 
September 10 
Lecture 02: Least Squares 
September 15 
Lecture 03: Linear Classification and Regression 
September 17 
Lecture 04: Neural Networks and BackProp 
September 22 
Lecture 05: Neural Networks and BackProp 
September 24 
Lecture 06: Support Vector Machines 
September 29 
Lecture 07: Support Vector Machines 
October 1 
Lecture 08: Kernels and Mappings 
October 6 
Lecture 09: Probability Models 
October 8 
Lecture 10: Probability Models 
October 13 
Lecture 11: Bernoulli Models and Naive Bayes 
October 15 
Lecture 12: Multinomial Models for Text 
October 20 
Lecture 13: Graphical Models Preview 
October 22 
Lecture 14: Gaussian Models 
October 27 
Lecture 15: Gaussian Regression and PCA 
October 29 
MIDTERM 
November 3 
ELECTION DAY (NO CLASS) 
November 5 
Lecture 16: Bayesian Inference 
November 10 
Lecture 17: The Exponential Family 
November 12 
Lecture 18: Mixture Models and Kmeans Clustering 
November 17 
Lecture 19: Expectation Maximization 
November 19 
Lecture 20: Expectation Maximization 
November 24 
Lecture 21: Graphical Models 
November 26 
THANKSGIVING DAY (NO CLASS) 
December 1 
Lecture 22: Graphical Models 
December 3 
Lecture 23: Junction Tree Algorithm 
December 8 
Lecture 24: Junction Tree Algorithm 
December 10 
Lecture 25: Hidden Markov Models 
December ?? 
COMPREHENSIVE FINAL EXAM 
Class
Attendance: You
are responsible for all material presented in the class
lectures,
recitations, and so forth. Some material will diverge from the textbooks
so
regular attendance is important.
Late
Policy: If you
hand in late work without approval of the instructor or TAs,
you will
receive zero credit. Deadlines are nonnegotiable.
Cooperation
on Homework:
Collaboration on solutions, sharing or copying of
solutions
is not allowed. Of course, no cooperation is allowed during exams.
This
policy will be strictly enforced.
Web
Page: The class
URL is: http://www.cs.columbia.edu/~jebara/4771
and
will
contain copies of class notes, news updates and other information.
Matlab: We'll use Matlab for coding, download it at www.cs.columbia.edu
by clicking on: > Computing > Software > Matlab.
Note: use JDK 1.6 instead of JDK 1.7.