ADVANCED MACHINE January, 2015

LEARNING & PERCEPTION

COMS 4772/6772 COURSE INFO

Day & Time and Location	M/W 1:10pm-2:25pm Location HAV 309
Instructor	Professor Tony Jebara jebara(at)cs(dot)columbia(dot)edu
Office Hours	CEPSR 605, Monday 2:30-3:30 or by appointment
TAs	Enze Li, el2742(at)columbia(dot)edu Office Hours: CEPSR 6LE5 Mon 10am-11am and TA Room Wed 10am-11am Kui Tang, kt2384(at)columbia(dot)edu Office Hours: CEPSR 6LE5 Tue 10am-11am or by appointment Sami Mourad, sm3891(at)columbia(dot)edu Office Hours: CEPSR 6LE5 Thu 10am-11am and Fri 11am-12pm

Prerequisites: COMS W4771 or permission of instructor. Knowledge of linear

algebra and introductory probability or statistics is required.

Description: 

Advanced topics in machine learning including: Linear Modeling,
Nonlinear Dimension Reduction, Maximum Entropy,
Exponential Family Models, Conditional Random Fields, Graphical
Models, Structured Support Vector Machines, Feature Selection, Kernel
Selection, Meta-Learning, Multi-Task Learning, Semi-Supervised
Learning, Graph-Based Semi-Supervised Learning, Approximate Inference,
Clustering, and Boosting.

Required Texts:

Primarily through handouts and links to various research papers.

Optional Texts:

Tony Jebara, Machine Learning: Discriminative and Generative.

Michael I. Jordan and Christopher M. Bishop, Introduction to Graphical Models.

Still unpublished. Available online (password-protected) on class home page.

R.O. Duda, P.E. Hart and D.G. Stork, Pattern Classification, John Wiley & Sons, 2001.

Trevor Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical

Learning. Springer Series in Statistics, Springer-Verlag New York USA. 2001.

Graded Work: Grades are based on 2 applied homeworks for 45% of the grade

and a large research level project with a final presentation (55%).

Tentative Schedule:

Date	Topics: A tentative wish list, we’ll see what we can go through!
Week 1	Introduction, Review of Basic Concepts, Representation Issues, Vector and Appearance-Based Models, Correlation and Least Squared Error Methods, Bases, Eigenspace Recognition, Principal Components Analysis
Week 2	Nonlinear Dimensionality Reduction, Manifolds, Kernel PCA, Locally Linear Embedding, Maximum Variance Unfolding, Minimum Volume Embedding
Week 3	Maximum Entropy, Exponential Families, Maximum Entropy Discrimination, Large Margin Probability Models
Week 4	Conditional Random Fields and Linear Models, Iterative Scaling and Majorization
Week 5	Graphical Models, Multi-Class Support Vector Machines, Structured Support Vector Machines, Cutting Plane Algorithms
Week 6	Kernels and Probabilistic Kernels
Week 7	Feature Selection and Kernel Selection, Support Vector Machine Extensions
Week 8	Meta-Learning and Multi-Task Support Vector Machines
Week 9	Semi-Supervised Learning and Graph-Based Semi-Supervised Learning
Week 10	High-Tree Width Graphical Models, Approximate Inference, Graph Structure Learning
Week 11	Clustering, Spectral Clustering, Normalized Cuts.
Week 12	Boosting, Mixtures of Experts, AdaBoost, Online Learning
Week 13	Project Presentations
Week 14	Project Presentations

Class Attendance: Class participation and interaction is an important aspect of this

course, ideally the course will run as a seminar where material presented in the class

lectures, recitations, and so forth. Some material will diverge from the textbooks

so regular attendance is important.

Late Policy: If you hand in late work without approval of the instructor or TAs,

you may receive zero credit. Homework is due as announced on its web page.

For the project, please submit on time regardless of additional progress.

For the final project, each day of lateness will cost you a minimum of 15%.

We won't give extensions, regardless of how amitious your project is.

Cooperation on Homework: Copying of solutions is forbidden.

Web Page: The class URL is: http://www.cs.columbia.edu/~jebara/4772 and

will contain copies of handouts, homework assignments, solutions and other

information.

Computer Accounts: You need a UNI account for courseworks.columbia.edu

and you will need access to Matlab for homeworks.