Math for Machine Learning

Objectives

The overall goal of this course is to strengthen the mathematical foundations that are required in pursuit of studying machine learning. This involves a good understanding of (i) linear algebra, (ii) multivariable calculus and optimization, and (iii) probability and statistics. This course assumes that the student has already taken courses in these subjects, and would like to be more comfortable with their mathematical maturity to be successful in machine learning and beyond.

• Mathematics for Machine Learning by Deisenroth, Faisal and Ong (link)
• Introduction to Applied Linear Algebra - Vectors, Matrices, and Least Squares by Boyd and Vandenberghe (link)
• Convex Optimization by Boyd and Vandenberghe (link)
• Introduction to Probability by Blitzstein and Hwang (link)

Administrative Details

• Assessment
• Homeworks (for practice only, must submit though, 0%)
  Quiz #0 (20%)
  Quiz #1 (20%)
  Quiz #2 (20%)
  Quiz #3 (20%)
  Class Participation (20%)
  
• Prerequisites
• Undergraduate courses in (i) Linear algebra, (ii) Multivariate calculus, (iii) Prob/Stats. This course is not a replacement for these vital courses!

Tentative Schedule

Module 1 - Linear Algebra

• Topic 1.1
• Introduction, Vectors, Matrices, and Least squares
  notes/reading: MML 2.1-2.8, 3.1-3.3 | VMLS 1.1-1.5, 2.1-2.3, 3.1-3.4, 5.1, 5.2, 6.1-6.4, 12.1-12.4
• Topic 1.2
• Subspaces, bases, and orthogonality
  notes/reading: MML 2.1-2.8, 3.1-3.3 | VMLS 1.1-1.5, 2.1-2.3, 3.1-3.4, 5.1, 5.2, 6.1-6.4, 12.1-12.4 | vid1 | vid2
• Topic 1.3
• Singular Value Decomposition
  notes/reading: MML 4.2, 4.4, 4.5
• Topic 1.4
• Eigendecomposition and Positive Semidefinite Matrices
  notes/reading: MML 4.2, 4.4, 4.5 | vid1

Module 2 - Multivariate Calculus and Optimization

• Topic 2.1
• Differentiation and vector calculus
  notes/reading: MML 5.1-5.5
• Topic 2.2
• Gradient Descent, Linearization, and Taylor Series
  notes/reading: MML 5.8 | vid1 | vid2
• Topic 2.3
• Optimization and the Lagrangian
  notes/reading: MML 7.1-7.2
• Topic 2.4
• Convexity and convex optimization
  notes/reading: MML 7.3 | CVX 1-3

Module 3 - Probability and Statistics

• Topic 3.1
• Probability Theory, Models, and Data
  notes/reading: MML 6.1-6.4 | BW 9
• Topic 3.2
• Bias, Variance and Statistical Estimators
  notes/reading: MML 6.1-6.4
• Topic 3.3
• The Central Limit Theorem, (some) "Named" Distributions, Multivariate Gaussian, and MLE
  notes/reading: MML 6.1-6.8, 8, 11

Policies

• Academic rules of conduct
• Students are expected to adhere to the Academic Honesty policy of the Computer Science Department, this policy can be found in full here.
• Violations
• Violation of any portion of these policies will result in a penalty to be assessed at the instructor's discretion. This may include receiving a zero grade for the assignment in question and a failing grade for the whole course, even for the first infraction.