Foundations of Graphical Models
Fall 2015
Columbia University
Course information
 Description and Syllabus
 Instructor: David
M. Blei
 Teaching Assistant: Maja Rudolph
 Meeting: Mondays and Wednesdays, 1:10PM2:25PM, 603 Hamilton
 Office hours for D. Blei: Wednesdays, 2:30PM4:30PM, CEPSR 703
 Office hours for M. Rudolph: 3:30PM4:30PM, Tuesdays (620 CEPSR)
and Thursdays (912 SSW)
 Piazza site
Course materials
Below are the readings and lecture notes. When available, we
include a link to the PDF of the readings. Otherwise, they are
available outside of Prof. Blei's office. The specific reading
assignments are announced on Piazza.
These topics may span multiple lectures in the class. See the
syllabus for the schedule. Note that these lecture notes are drafts
and works in progress. Feel free to email david.blei@columbia.edu
with comments and errors.
 Introduction
 A Quick Review of Probability
 Basics of Graphical
Models
 Reading: "Conditional Independence and Factorization" in
Introduction to Probabilistic Graphical Models (Jordan, 2003).
 Elimination, Tree Propagation, and the Hidden Markov Model
 Reading: "The Elimination Algorithm" in Introduction to
Probabilistic Graphical Models (Jordan, 2003)
 Reading: "Probability Propagation and Factor Graphs" in
Introduction to Probabilistic Graphical Models (Jordan, 2003)
 Models, data, and
statistical concepts
 Bayesian Mixture
Models and the Gibbs Sampler
 Probabilistic Modeling in Stan
 Exponential Families and Conjugate Priors
 Reading: "The Exponential Family" (Bishop, 2006; Section 2.4)
 Mixedmembership Models and MeanField Variational Inference
 Matrix Factorization and Recommendation Systems
 Generalized Linear Models
 Reading: "An outline of generalized linear models" in
Generalized Linear Models (McCullough and Nelder, 1989)
 Reading: "Multilevel linear structures" in Data Analysis Using Regression and Multilevel/Hierarchical Models
(Gelman and Hill, 2007)
 Reading: "Multilevel linear models: The basics" in Data Analysis Using Regression and Multilevel/Hierarchical Models
(Gelman and Hill, 2007)
 Regularized Regression
 Reading: The Elements of Statistical
Learning, Chapters 3.1, 3.2, 3.4, 3.6 (Hastie et al., 2009)
 Bayesian Nonparametric Models
Homework assignments

Homework 1
Out: 20150930
Due: 20151012

Homework 2
Out: 20151020
Due: 20151104
Other materials