\documentclass[12pt]{article}
\usepackage {fullpage}
\usepackage {palatino}
\usepackage {url}
%\textwidth 6.5in
%\topmargin 0in
%\headheight 0in
%\headsep 0.5in
%\textheight 8.5in
%\oddsidemargin 0in
%\evensidemargin 0in
\parskip 0.1in
\parindent 0in

%\newenvironment{itemizesp}{\bgroup\samepage\begin{itemize}}{\end{itemize}\egroup}
\newenvironment{itemizesp}{\begin{itemize}}{\end{itemize}}
\newenvironment{outdent}[1]%
{\setbox1=\hbox{#1}
 \begin{list}{}%
 {\setlength{\labelwidth}{\wd1}
  \setlength{\leftmargin}{\wd1}
  \addtolength{\leftmargin}{1em}
  \addtolength{\leftmargin}{\labelsep}
  \setlength{\rightmargin}{1em}
}}%
{\end{list}}

\newcommand{\litem}[1]{\item[#1\hfill]}
\newcommand{\ritem}[1]{\item[#1]}
\newcommand{\cut}[1]{}

\setcounter{secnumdepth}{0}

\begin{document} 

% \vspace{.2in}

\begin{center}
%{\Large CSC 411}\\
%\vspace*{0.1in}
%{\Large MACHINE LEARNING and DATA MINING}
{\Large COMS 4995: Neural Networks and Deep Learning}\\
\vspace*{0.1in}
{\large Instructor: Richard Zemel}
\end{center}

\begin{tabbing}
Lectures: \hspace*{1.in} \= Monday, Wednesday 2:40-3:55\\
%Lecture Room: \> CSB 451\\
Instructor: \> Richard Zemel\\% ~$<$csc411prof@cs.toronto.edu$>$\\
%Office hours: \>  Monday 1:30-2:30 CEPSR 605, and by appointment\\
%%%%Teaching Assistants:\> Feyyaz Demir, Aziz Mezlini, Jake Snell, and Nitish Srivastava \\
%TA email:\> $<$csc411ta@cs.toronto.edu$>$ \\
%Tutorials: \> Friday 3-4\\
%Tutorial Room: \> BA 1190\\
%Class URL: \> www.cs.toronto.edu/\textasciitilde zemel/Courses/CS411.html\\
\end{tabbing}

\section*{Overview}
It is very hard to hand design programs to solve many real world problems, e.g. distinguishing
images of cats versus dogs. Machine learning algorithms allow computers to learn from example
data, and produce a program that does the job. Neural networks are a class of machine learning
algorithm originally inspired by the brain, but which have recently have seen a lot of success at
practical applications. They're at the heart of production systems at companies like Google and
Facebook for image processing, speech-to-text, and language understanding. This course gives an
overview of both the foundational ideas and the recent advances in neural net algorithms.

Roughly the first 2/3 of the course focuses on supervised learning -- training the network to
produce a specified behavior when one has lots of labeled examples of that behavior. The last 1/3
focuses on unsupervised learning and reinforcement learning.

\section*{Pre-requisites}

This is a second course in machine learning, so it has some substantial
prerequisites. 
Required courses: Machine Learning; Multivariable Calculus; Linear Algebra; Probability \& Statistics.
These
prerequisites will not be enforced, but without them the course will be
extremely challenging.
%\begin{itemize}
%\item  Multivariable Calculus
%\item Linear Algebra
%\item Machine Learning
%\end{itemize}

\section*{Readings}

There is no required textbook for the class. A few small readings may be
assigned if the need
arises. These required readings will all be available on the web, for free.

There are also some relevant resources which are freely available online. We will try to provide
links on a lecture-by-lecture basis.
\begin{itemize}
\item Video lectures for UofT Professor Geoffrey Hinton's Coursera course. Professor Hinton is one
of the fathers of the field, so think of these as the Feynman Lectures of neural nets.\\
\url{ https://www.youtube.com/playlist?list=PLoRl3Ht4JOcdU872GhiYWf6jwrkSNhz9}
\item {\em Deep Learning}, a textbook by Yoshua Bengio, Ian Goodfellow, and Aaron Courville.
\url{ http://www.deeplearningbook.org/}
\item Andrej Karpathy's lecture notes on convolutional networks. These are very readable and
cover the material in roughly the first half of the course.
\url{ http://cs231n.github.io/}
\item Richard Socher's lecture notes, focusing on RNNs.
\url{ http://cs224d.stanford.edu/syllabus.html}
\item Metacademy, an online website which helps
you construct personalized learning plans and which has links to lots of resources relevant
to particular concepts. We'll post links to relevant Metacademy concepts as the course pro-
gresses.
\url{ http://www.metacademy.org}
\item Video lectures for Hugo Larochelle's neural networks course. These are similar to Professor
Hinton's lectures but a bit more mathematical.
\url{ http://info.usherbrooke.ca/hlarochelle/neural networks/content.html}
\item David MacKay's excellent textbook, {\em Information Theory, Inference, and Learning Algorithms.}
This isn't focused on neural nets per se, but it has some overlap with this course, especially
the lectures on Bayesian models.
\url{ http://www.inference.phy.cam.ac.uk/mackay/itila/}
\item {\em Neural Networks and Deep Learning}, a book by physicist Michael Nielsen which covers the
basics of neural nets and backpropagation.\\
\url{ http://neuralnetworksanddeeplearning.com/}
\end{itemize}

\section*{Course requirements and grading}

The format of the class will be lecture, with some discussion. I
strongly encourage interaction and questions.  There are assigned
readings for each lecture that are intended to prepare you to
participate in the class discussion for that day.

The grading in the class will be divided up as follows:
\begin{center}
\begin{tabular}{ll}
3 Assignments (Programming \& Written)            & 54\% \\
Mid-Term Test           & 20\% \\
Project              & 25\% \\
Attendance \& Participation & 1\%
%Final Exam           & 28\% \\
\end{tabular}
\end{center}

% There will be a total of 4 programming assignments.
% And 4 written assignments. For these your three highest marks will count for 10\% each
% (lowest mark will be dropped).
% 
% The marking scheme for graduate students is identical to the marking scheme for undergraduates,
% except that the midterm test exam is weighted towards the final project. The requirements and
% marking scheme for the final project will be posted separately on the course web page.

\clearpage
 
\begin{center}
%{\Large \bf CLASS SCHEDULE\\[.1in](subject to change)}
{\Large \bf CLASS SCHEDULE}
\end{center}

Shown below are the topics for lectures and tutorials. These are
subject to change.
The notes will be available on the
class web-site the day of the class meeting. 
%Specific readings for
%each lecture will be posted on the website as well.

\vspace{-.1in}
\begin{center}
\begin{tabular}{|l|l|}
\hline &\\[-.1in]
\hspace*{.1in} {\large \bf Session} & \hspace*{.65in} {\large \bf Topic}\\[.01in]
\hline &\\[-.1in]
Lecture 1 & Introduction \& Linear Models\\[.01in]
\hline &\\[-.1in]
{\em Tutorial 1} & {\em Multivariable Calculus Review} \\[.01in]
\hline &\\[-.1in]
Lecture 2 & Multilayer Perceptrons \& Backpropagation \\[.01in]
\hline &\\[-.1in]
{\em  Tutorial 2} &  {\em Autograd and PyTorch}\\[.01in]
\hline &\\[-.1in]
Lecture 3 & Distributed Representations \& Optimization \\[.01in]
\hline &\\[-.1in]
{\em Tutorial 3} & {\em How to Train Neural Networks} \\[.01in]
\hline &\\[-.1in]
Lecture 4 & Convolutional Neural Networks and Image Classification\\[.01in]
\hline &\\[-.1in]
{\em Tutorial 4} & {\em Convolutional Neural Networks} \\[.01in]
\hline &\\[-.1in]
Lecture 5 &	Interpretability  \\[.01in]
\hline &\\[-.1in]
Midterm Quiz \\[.01in]
\hline &\\[-.1in]
Lecture 6 & Optimization \& Generalization \\[.01in]
\hline &\\[-.1in]
{\em Tutorial 6} & {\em Best Practices of ConvNet Applications}  \\[.01in]
\hline &\\[-.1in]
Lecture 7 & Recurrent Neural Networks and Attention \\[.01in]
\hline &\\[-.1in]
{\em Tutorial 7} & {\em Recurrent Neural Networks}\\[.01in]
\hline &\\[-.1in]
Lecture 8 & Transformers and Autoregressive Models\\[.01in]
\hline &\\[-.1in]
{\em Tutorial 8} & {\em NLP and Transformers}\\[.01in]
\hline &\\[-.1in]
Lecture 9 & Reversible Models \& Generative Adversarial Networks\\[.01in]
\hline &\\[-.1in]
{\em Tutorial 9} & {\em Information Theory}\\[.01in]
\hline &\\[-.1in]
Lecture 10 & Generative Models \& Reinforcement Learning \\[.01in]
\hline &\\[-.1in]
{\em Tutorial 10} & {\em Generative Adversarial Networks}\\[.01in]
\hline &\\[-.1in]
Lecture 11 & Q-learning \& the Game of Go \\[.01in]
\hline &\\[-.1in]
{\em Tutorial 11} & {\em Policy Gradient and Reinforcement Learning} \\[.01in]
\hline &\\[-.1in]
Lecture 12 & Recent Trends in Deep Learning\\[.01in]
\hline 
\end{tabular}
\end{center}
\end{document}
