COMS 6998-4: Learning and Empirical Inference
Irina Rish, Gerry Tesauro, Tony Jebara and Vladimir Vapnik
Room 644 MUDD
Irina and Gerry's Office Hours Tue 12:00-1:00PM in Room 644 MUDD
Tony's Office Hours Mon/Wed 04:15-04:45PM in Room 605 CEPSR
Advanced topics for students interested in research in machine
learning. Empirical Inference Science, Transduction, Universum, SVM+,
Semidefinite Programming, Improved VC Bounds, Ellipsoid Machines,
Fisher Discriminant, Metric Learning, Nearest Components Analysis, Xing
Clustering with Side Info, Supervised Dimensionality Reduction, Metric
Learning & Regression, Probabilistic connections to distances and
Bregman divergences, Bayes point machines, E-Family PCA, Sparse
optimization, sparse PCA, feature selection, matrix factorization,
l1-norm SVMs, Structured prediction, Max Margin Markov Nets,
Conditional Random Fields, Reinforcement Learning, Active learning,
Model Selection, Occam Razor, BIC, AIC, Information Bottleneck Online
Learning, Boosting, Bounds, Sampling, Invariance, Groups.
Jan 16, I&G: Overview and hot topics in Machine Learning, Reinforcement Learning.
Jan 23, T: Improved SVM VC Bounds, Semidefinite Programming, Minimum Volume Ellipsoids.
Jan 30, V: Induction and Empirical Inference.
Feb 06, I: Active Learning.
Feb 13, G: Reinforcement Learning Applications. Game Theory. Multi-Agent learning.
Feb 20, T: Learning and graphs (connecting the dots).
Feb 27: CLASS CANCELLED.
Mar 06, V: Transduction, Universum and SVM+.
Mar 13, I: Bregman divergence and exponential family PCA. ... and Java Demo
Mar 20, G: Metric learning. Part 1. Part 2. Sam's Notes. Yann's Notes.
Mar 27, T: Structured prediction, max margin Markov networks and CRFs.
Apr 03, V: Project Discussions and Brainstorming.
Apr 10, I: Sparse and l1-regularization.
Apr 17: CLASS CANCELLED. Will be rescheduled.
Apr 24: Most Project Presentations.
May 01: Last of the Presentations.
May 01: Energy Based Modeling + Invariance and Permutation.
May 04: Papers due by email to the 4 instructors.
100% of the grade will be based on a conference-style novel paper and presentation.
Feel free to email the instructors and TA's for help with your papers.
To see how to write a good paper and present it, check out this link:
In particular see Simon
Peyton Jones on "How to Write a Good Research Paper".
Follow this link for the 2007 presentations.
Irina Rish is a Research Staff Member at IBM
T. J. Watson Research Center. She received MS in Applied Mathematics
from Moscow Gubkin Institute, Russia, and PhD in Computer Science from
the University of California, Irvine. Irina's primary research
interests are in the areas of probabilistic inference, machine
learning, and information theory. Particularly, she has been working
on approximate inference in probabilistic graphical models,
information-theoretic experiment design, active learning, and their
applications to the area of autonomic computing, or automated
management of complex distributed systems, which includes various
iagnosis, prediction and online decision-making problems. She was also
teaching several machine-learning courses at EE Department of Columbia
University as an adjunct professor.
Gerry Tesauro is a Research Staff Member at IBM T. J. Watson Research
Center. He received a PhD in theoretical physics from Princeton
University in 1986, and subsequently converted to machine learning
research after being swept up in the neural networks craze of that
era. In his career at IBM he has worked on theoretical and applied
machine learning in wide variety of settings, including multi-agent
learning, dimensionality reduction, credit scoring, computer virus
recognition, computer chess (Deep Blue), intelligent e-commerce agents,
and most notoriously, TD-Gammon, a self-teaching program that learned
to play backgammon at human world championship level. He has also been
heavily involved for many years in the annual NIPS conference, and was
NIPS Program Chair in 1993, General Chair in 1994, and is a member of
the NIPS Foundation Board of Directors. He is currently interested
in exploring potential wide applicability of ML approaches throughout
the huge emerging domain of self-managing computing systems.
Tony Jebara is an Assistant Professor of Computer Science at Columbia
University. He is Director of the Columbia Machine Learning Laboratory
whose research focuses upon machine learning, computer vision and
related application areas such as human-computer interaction. Jebara
is also a Principal Investigator at Columbia's Vision and Graphics
Center. He has published over 40 peer-reviewed papers in conferences
and journals including NIPS,
ICML, UAI, COLT, JMLR, CVPR, ICCV, and PAMI. He is the author
of the book Machine Learning: Discriminative and Generative
(Kluwer). Jebara is the recipient of the Career award from the
National Science Foundation and has also received honors for his
papers from the International Conference on Machine Learning and from
the Pattern Recognition Society. He has served as chair,
program committee member and reviewer for various conferences and
workshops. Jebara's research has been featured on television (ABC,
BBC, New York One, TechTV, etc.) as well as in the popular press
(Wired Online, Scientific American, Newsweek, Science Photo Library,
etc.). Jebara obtained his Bachelor's from McGill University (at the
McGill Center for Intelligent Machines) in 1996. He obtained his
Master's in 1998 and his PhD in 2002 both from the Massachusetts
Institute of Technology (at the MIT Media Laboratory). He is currently
a member of the IEEE, ACM and AAAI. Professor Jebara's research and
laboratory are supported in part by the Central Intelligence Agency,
Microsoft, Alpha Star Corporation and the National Science Foundation.
Vladimir Naumovich Vapnik is one of the main developers of
Vapnik-Chervonenkis theory. He was born in the Soviet Union; received
a master's degree in mathematics from the Uzbek State University in
Samarkand (now Uzbekistan), in 1958; and received a Ph.D in statistics
from the Institute of Control Science in Moscow in 1964. He worked at
this institute from 1961 until 1990, and became Head of the Computer
Science Research Department. In 1995 he was appointed Professor of
Computer Science and Statistics at Royal Holloway, University of
London. At AT&T Bell Labs (later Shannon Labs) from 1991 through 2001,
Vapnik and his colleagues developed the theory of the support vector
machine. They demonstrated its performance on a number of problems of
interest to the machine learning community, including handwriting
recognition. He is currently at NEC Laboratories in Princeton, New
Jersey, and also Columbia University, New York, New York. In 2006,
Vapnik was inducted into the U.S. National Academy of Engineering.