MACHINE LEARNING January 18, 2011

COMS4771-001 COURSE INFO

Day & Time and Location	T/Th 2:40pm-3:55pm at 309 Havemeyer
Instructor	Professor Tony Jebara, jebara(at)cs(dot)columbia(dot)edu
Office Hours	T/Th 4:00pm-4:45pm at 605 CEPSR
TAs	Ido Rosen, ir2002(at)columbia(dot)edu Tsung-Kai, Lin, tl2450(at)columbia(dot)edu Chun Li, cl2894(at)columbia(dot)edu Adrian Weller, aw2506(at)columbia(dot)edu
Bulletin Board	Available via courseworks.columbia.edu and is the best method of contact between students and Professor/TAs for general questions that would be relevant for the whole class (i.e. clarifications on lectures, questions about homework, etc.).

Prerequisites: Knowledge of linear algebra and introductory probability or statistics.

Description: This course introduces topics in machine learning for both generative
and discriminative estimation. Material will include least squares methods, Gaussian
distributions, linear classification, linear regression, maximum likelihood, exponential
family distributions, Bayesian networks, Bayesian inference, mixture models, the EM
algorithm, graphical models, hidden Markov models, support vector machines, and
kernel methods. Students are expected to implement several algorithms in Matlab
and have some background in linear algebra and statistics.

Required Texts:

Michael I. Jordan and Christopher M. Bishop, Introduction to Graphical Models.

Still unpublished. Available online (password-protected) on class home page.

Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer.

2006 First Edition is preferred. ISBN: 0387310738. 2006.

Optional Texts: Available at library (additional handouts will also be given).

Tony Jebara, Machine Learning: Discriminative and Generative, Kluwer, 2004

ISBN: 1-4020-7647-9. Boston, MA, 2004.

R.O. Duda, P.E. Hart and D.G. Stork, Pattern Classification, John Wiley & Sons, 2001.

Trevor Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical

Learning. Springer Series in Statistics, Springer-Verlag New York USA. 2001.

Tom M. Mitchell, Machine Learning, McGraw-Hill Series in Computer Science, 1997.

Graded Work: Grades will be based on 5 homeworks (about 50%), the midterm (about

20%) and the final exam (about 30%). Any material covered in assigned book

readings, handouts, homework, lectures or discussion sections may appear in exam

The midterm is the last class before Spring Break.

If you miss the midterm and don't have an official reason, you will get 0 on it.

If you have an official reason, your midterm grade will be based on the final exam.

Tentative Schedule:

Date	Topic
January 18	Lecture 01: Introduction
January 20	Lecture 02: Least Squares
January 25	Lecture 03: Linear Classification and Regression
January 27	Lecture 04: Neural Networks and BackProp
February 1	Lecture 05: Neural Networks and BackProp
February 3	Lecture 06: Support Vector Machines
February 8	Lecture 07: Support Vector Machines
February 10	Lecture 08: Kernels and Mappings
February 15	Lecture 09: Probability Models
February 17	Lecture 10: Probability Models
February 22	Lecture 11: Bernoulli Models and Naive Bayes
February 24	Lecture 12: Multinomial Models for Text
March 1	Lecture 13: Graphical Models Preview
March 3	Lecture 14: Gaussian Models and Estimation
March 8	Lecture 15: Gaussian Classification, Regression, PCA
March 10	MIDTERM
March 15	SPRING BREAK
March 17	SPRING BREAK
March 22	Lecture 16: Bayesian Inference
March 24	Lecture 17: The Exponential Family
March 29	Lecture 18: Mixture Models and Clustering
March 31	Lecture 19: Expectation Maximization
April 5	Lecture 20: Expectation Maximization
April 7	Lecture 21: Graphical Models
April 12	Lecture 22: Graphical Models
April 14	Lecture 23: Graphical Models
April 19	Lecture 24: Junction Tree Algorithm
April 21	Lecture 25: Junction Tree Algorithm
April 26	Lecture 26: Hidden Markov Models (HMMs)
April 28	Lecture 27: HMMs and Structure Learning

Class Attendance: You are responsible for all material presented in the class

lectures, recitations, and so forth. Some material will diverge from the textbooks

so regular attendance is important.

Late Policy: If you hand in late work without approval of the instructor or TAs,

you will receive zero credit. Homework is due at the beginning of class on the

due date.

Cooperation on Homework: Collaboration on solutions, sharing or copying of

solutions is not allowed. Of course, no cooperation is allowed during exams.

This policy will be strictly enforced.

Web Page: The class URL is: http://www.cs.columbia.edu/~jebara/4771 and

will contain copies of handouts, homework assignments, solutions and other

information.

Computer Accounts: You will need an ACIS computer account for email, use

of Matlab (unless you have a windows version) and so forth.