MACHINE LEARNING                             September 8, 2015



Time & Location

T/Th 10:10am-11:25am at TBD


Professor Tony Jebara, jebara(at)cs(dot)columbia(dot)edu

Office Hours

W 2:00pm-4:00pm at 605 CEPSR


Ayushi Singhal, as4521(at)columbia(dot)edu

Wenting Yin, wy2214(at)columbia(dot)edu

Ruichi Yu, ry2254(at)columbia(dot)edu

Sarper Sertoglu, ss4552(at)columbia(dot)edu

Michael Yang, my2292(at)columbia(dot)edu

Bulletin Board

Available via and is the best
way to contact the Professor and the TAs. Use it for
clarifications on lectures, questions about homework, etc.


Prerequisites: Knowledge of linear algebra and introductory probability or statistics.


Description: This course introduces topics in machine learning for both generative
and discriminative estimation. Material will include least squares methods, Gaussian
distributions, linear classification, linear regression, maximum likelihood, exponential
family distributions, Bayesian networks, Bayesian inference, mixture models, the EM
algorithm, graphical models, hidden Markov models, support vector machines, and
kernel methods. Students are expected to implement several algorithms in Matlab
and have some background in linear algebra and statistics.


Required Texts:


Michael I. Jordan and Christopher M. Bishop, Introduction to Graphical Models.

Still unpublished. Available online via


Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer.

2006 First Edition is preferred. ISBN: 0387310738. 2006.


Optional Texts: Available at library (additional handouts will also be given).


Tony Jebara, Machine Learning: Discriminative and Generative, Kluwer, 2004

ISBN: 1402076479. Boston, MA, 2004.


R.O. Duda, P.E. Hart and D.G. Stork, Pattern Classification, John Wiley & Sons, 2001.


Trevor Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical

Learning. Springer-Verlag New York USA, 2009. 2nd Edition. ISBN 0387848576.


Graded Work: Grades will be based on 5 homeworks (45%), the midterm (20%),

two surprise in-class quizzes (5%), and the final exam (30%). Any material covered in

assigned readings, handouts, homeworks, solutions, or lectures may appear in exams.

If you miss the midterm and don't have an official reason, you will get 0 on it.

If you have an official reason, your midterm grade will be based on the final exam.

If you miss a quizz and don't have an official reason, you will get 0 on it.

If you have an official reason, your missed quiz grade will be based on the final exam.


Tentative Schedule:



September 2

Lecture 01: Introduction

September 4

Lecture 02: Least Squares

September 9

Lecture 03: Linear Classification and Regression

September 11

Lecture 04: Neural Networks and BackProp

September 16

Lecture 05: Neural Networks and BackProp

September 18

Lecture 06: Support Vector Machines

September 23

Lecture 07: Support Vector Machines

September 25

Lecture 08: Kernels and Mappings

September 30

Lecture 09: Probability Models

October 2

Lecture 10: Probability Models

October 7

Lecture 11: Bernoulli Models and Naive Bayes

October 9

Lecture 12: Multinomial Models for Text

October 14

Lecture 13: Graphical Models Preview

October 16

Lecture 14: Gaussian Models

October 21

Lecture 15: Gaussian Regression and PCA

October 23


October 28

Lecture 16: Bayesian Inference

October 30

Lecture 17: The Exponential Family

November 4


November 6

Lecture 18: Mixture Models and Kmeans Clustering

November 11

Lecture 19: Expectation Maximization

November 13

Lecture 20: Expectation Maximization

November 18

Lecture 21: Graphical Models

November 20

Lecture 22: Graphical Models

November 25

Lecture 23: Junction Tree Algorithm

November 27


December 2

Lecture 24: Junction Tree Algorithm

December 4

Lecture 25: Hidden Markov Models

December ??




Class Attendance: You are responsible for all material presented in the class

lectures, recitations, and so forth. Some material will diverge from the textbooks

so regular attendance is important.


Late Policy: If you hand in late work without approval of the instructor or TAs,

you will receive zero credit. Deadlines are non-negotiable.


Cooperation on Homework: Collaboration on solutions, sharing or copying of

solutions is not allowed. Of course, no cooperation is allowed during exams.

This policy will be strictly enforced.


Web Page: The class URL is: and

will contain copies of class notes, news updates and other information.


Matlab: We'll use Matlab for coding, download it at

by clicking on: -> Computing -> Software -> Matlab.

Note: use JDK 1.6 instead of JDK 1.7.