COMS 6998-4: Learning and Empirical Inference
Irina Rish, Gerry Tesauro, Tony Jebara and Vladimir Vapnik
http://www.cs.columbia.edu/~jebara/6998

Tuesdays 1:00PM-2:50PM
Room 644 MUDD


Irina and Gerry's Office Hours Tue 12:00-1:00PM in Room 644 MUDD
Tony's Office Hours Mon/Wed 04:15-04:45PM in Room 605 CEPSR


SYLLABUS
Advanced topics for students interested in research in machine learning. Empirical Inference Science, Transduction, Universum, SVM+, Semidefinite Programming, Improved VC Bounds, Ellipsoid Machines, Fisher Discriminant, Metric Learning, Nearest Components Analysis, Xing Clustering with Side Info, Supervised Dimensionality Reduction, Metric Learning & Regression, Probabilistic connections to distances and Bregman divergences, Bayes point machines, E-Family PCA, Sparse optimization, sparse PCA, feature selection, matrix factorization, l1-norm SVMs, Structured prediction, Max Margin Markov Nets, Conditional Random Fields, Reinforcement Learning, Active learning, Model Selection, Occam Razor, BIC, AIC, Information Bottleneck Online Learning, Boosting, Bounds, Sampling, Invariance, Groups.

SCHEDULE
Jan 16, I&G: Overview and hot topics in Machine Learning, Reinforcement Learning.
Jan 23, T: Improved SVM VC Bounds, Semidefinite Programming, Minimum Volume Ellipsoids.
Jan 30, V: Induction and Empirical Inference.
Feb 06, I: Active Learning.
Feb 13, G: Reinforcement Learning Applications. Game Theory. Multi-Agent learning.
Feb 20, T: Learning and graphs (connecting the dots).
Feb 27: CLASS CANCELLED.
Mar 06, V: Transduction, Universum and SVM+.
Mar 13, I: Bregman divergence and exponential family PCA. ... and Java Demo
Mar 20, G: Metric learning. Part 1. Part 2. Sam's Notes. Yann's Notes.
Mar 27, T: Structured prediction, max margin Markov networks and CRFs.
Apr 03, V: Project Discussions and Brainstorming.
Apr 10, I: Sparse and l1-regularization.
Apr 17: CLASS CANCELLED. Will be rescheduled.
Apr 24: Most Project Presentations.
May 01: Last of the Presentations.
May 01: Energy Based Modeling + Invariance and Permutation.
May 04: Papers due by email to the 4 instructors.


GRADING
100% of the grade will be based on a conference-style novel paper and presentation.
Feel free to email the instructors and TA's for help with your papers.
To see how to write a good paper and present it, check out this link:
http://www.cs.iastate.edu/~honavar/grad-advice.html
In particular see Simon Peyton Jones on "How to Write a Good Research Paper".

PRESENTATIONS
Follow this link for the 2007 presentations.

BIOGRAPHIES

Irina Rish is a Research Staff Member at IBM T. J. Watson Research Center. She received MS in Applied Mathematics from Moscow Gubkin Institute, Russia, and PhD in Computer Science from the University of California, Irvine. Irina's primary research interests are in the areas of probabilistic inference, machine learning, and information theory. Particularly, she has been working on approximate inference in probabilistic graphical models, information-theoretic experiment design, active learning, and their applications to the area of autonomic computing, or automated management of complex distributed systems, which includes various iagnosis, prediction and online decision-making problems. She was also teaching several machine-learning courses at EE Department of Columbia University as an adjunct professor.

Gerry Tesauro is a Research Staff Member at IBM T. J. Watson Research Center. He received a PhD in theoretical physics from Princeton University in 1986, and subsequently converted to machine learning research after being swept up in the neural networks craze of that era. In his career at IBM he has worked on theoretical and applied machine learning in wide variety of settings, including multi-agent learning, dimensionality reduction, credit scoring, computer virus recognition, computer chess (Deep Blue), intelligent e-commerce agents, and most notoriously, TD-Gammon, a self-teaching program that learned to play backgammon at human world championship level. He has also been heavily involved for many years in the annual NIPS conference, and was NIPS Program Chair in 1993, General Chair in 1994, and is a member of the NIPS Foundation Board of Directors. He is currently interested in exploring potential wide applicability of ML approaches throughout the huge emerging domain of self-managing computing systems.

Tony Jebara is an Assistant Professor of Computer Science at Columbia University. He is Director of the Columbia Machine Learning Laboratory whose research focuses upon machine learning, computer vision and related application areas such as human-computer interaction. Jebara is also a Principal Investigator at Columbia's Vision and Graphics Center. He has published over 40 peer-reviewed papers in conferences and journals including NIPS, ICML, UAI, COLT, JMLR, CVPR, ICCV, and PAMI. He is the author of the book Machine Learning: Discriminative and Generative (Kluwer). Jebara is the recipient of the Career award from the National Science Foundation and has also received honors for his papers from the International Conference on Machine Learning and from the Pattern Recognition Society. He has served as chair, program committee member and reviewer for various conferences and workshops. Jebara's research has been featured on television (ABC, BBC, New York One, TechTV, etc.) as well as in the popular press (Wired Online, Scientific American, Newsweek, Science Photo Library, etc.). Jebara obtained his Bachelor's from McGill University (at the McGill Center for Intelligent Machines) in 1996. He obtained his Master's in 1998 and his PhD in 2002 both from the Massachusetts Institute of Technology (at the MIT Media Laboratory). He is currently a member of the IEEE, ACM and AAAI. Professor Jebara's research and laboratory are supported in part by the Central Intelligence Agency, Microsoft, Alpha Star Corporation and the National Science Foundation.

Vladimir Naumovich Vapnik is one of the main developers of Vapnik-Chervonenkis theory. He was born in the Soviet Union; received a master's degree in mathematics from the Uzbek State University in Samarkand (now Uzbekistan), in 1958; and received a Ph.D in statistics from the Institute of Control Science in Moscow in 1964. He worked at this institute from 1961 until 1990, and became Head of the Computer Science Research Department. In 1995 he was appointed Professor of Computer Science and Statistics at Royal Holloway, University of London. At AT&T Bell Labs (later Shannon Labs) from 1991 through 2001, Vapnik and his colleagues developed the theory of the support vector machine. They demonstrated its performance on a number of problems of interest to the machine learning community, including handwriting recognition. He is currently at NEC Laboratories in Princeton, New Jersey, and also Columbia University, New York, New York. In 2006, Vapnik was inducted into the U.S. National Academy of Engineering.