Rocco's lecture notes will be posted soon after the class. You can also find course videos in the "Video Library" section of Courseworks soon after class.
Warning: the notes below were generated in real time and have not been edited. They may contain typos or other errors. They may also contain aesthetically displeasing color combinations.
|1||Wed Sept 6||Introduction, basics||Blum survey sec. 3.0|
|2||Mon Sept 11||Online mistake-bound learning, elimination algorithm, decision lists||Blum survey sec. 3.0, 3.1|
|3||Wed Sept 13||Learning decision lists, Winnow1||Blum survey sec. 3.2, Littlestone paper sec. 5 (just through Theorem 7)|
|4||Mon Sept 18||Winnow2, Perceptron||Blum survey sec. 3.2, Littlestone paper sec. 5 (just through Theorem 7), handout on Perceptron and kernel methods|
|5||Wed Sept 20||Perceptron, dual Perceptron, kernel methods||handout on Perceptron and kernel methods||6||Mon Sept 25||General bounds on OLMB learning: Halving Algorithm, Randomized H.A., start VC Dimension||Blum survey sec. 2.0, 2.1, 2.2, Littlestone paper sec. 1-3 (don't worry about the stuff about the SOA)|
|7||Wed Sept 27||General bounds on OLMB learning: VC dimension, Weighted Majority algorithm||Blum survey sec. 2.0, 2.1, 2.2, Littlestone paper sec. 1-3 (don't worry about the stuff about the SOA)|
|8||Mon Oct 2||Randomized Weighted Majority algorithm, intro to PAC learning, PAC learning intervals||Kearns and Vazirani chapter 1.1-1.3|
|9||Wed Oct 4||finish PAC learning intervals, OLMB to PAC conversion, definitional issues||Kearns and Vazirani chapter 1.1-1.3|
|10||Mon Oct 9||Chernoff bounds, learning by finding consistent hypotheses, Occam's Razor||Kearns and Vazirani chapters 1,2, appendix (Chapter 9), this handout on probability basics, this handout on Chernoff bounds|
|11||Wed Oct 11||PAC sample-efficient learning sparse disjunctions via Occam and greedy set cover, start proper versus improper learning||Kearns and Vazirani chapters 1,2|
|12||Mon Oct 16||Improper PAC learning of 3-term DNF is computationally easy, proper PAC learning of 3-term DNF is computationally hard||Kearns and Vazirani chapters 1,2|
|13||Wed Oct 18||Finish hardness of proper PAC learning 3-term DNF; Lower bound on PAC learning sample complexity based on VC dimension; start upper bound||Kearns and Vazirani chapter 3|
|14||Mon Oct 23||No lecture (midterm exam)||15||Wed Oct 25||Upper bound on PAC learning sample complexity based on VC dimension: Sauer-Shelah-Perles lemma||Kearns and Vazirani chapter 3||15||Mon Oct 30||Upper bound on PAC learning sample complexity based on VC dimension: ``double sample'' argument, application to PAC learning LTFs over \R^n||Kearns and Vazirani chapter 3 (you can peek at Chapter 4.0-4.3.2 as a head start for next time)||16||Wed Nov 1||Confidence boosting; accuracy boosting overview; start simple 3-stage accuracy improving procedure||Kearns and Vazirani Chapter 4.0-4.3.2||Mon Nov 6||No lecture (University holiday)||17||Wed Nov 8||Finish simple 3-stage accuracy improving procedure; boosting over a fixed sample; AdaBoost||PDF, annotated AdaBoost algorithm||Kearns and Vazirani Chapter 4.0-4.3.2; clean AdaBoost handout; Schapire boosting overview paper||18||Thurs Nov 10||AdaBoost analysis; start PAC learning with noise||Schapire boosting overview paper||19||Wed Nov 15||PAC learning with malicious noise; start PAC learning with random classification noise||Kearns and Vazirani Chapter 5||20||Mon Nov 20||PAC learning with random classification noise; Statistical Query learning||Kearns and Vazirani Chapter 5||21||Mon Nov 27||Statistical Query learning algorithms yield RCN-tolerant PAC algorithms; start lower bounds on SQ learning||Kearns and Vazirani Chapter 5||22||Wed Nov 29||Lower bounds on SQ learning; start cryptographic hardness of learning||Kearns and Vazirani Chapter 5||23||Mon Dec 4||Cryptographic hardness of learning based on pseudorandomness, start crypto hardness based on PKC||Kearns and Vazirani Chapter 6||24||Wed Dec 6||Cryptographic hardness of learning based on PKC / trapdoor one-way permutations, discrete cube roots||Kearns and Vazirani Chapter 6||25||Thurs Dec 8||Cryptographic hardness of learning simple circuits based on discrete cube roots, peek at other topics||Kearns and Vazirani Chapter 6|
Here is an anticipated list of topics. Note that the ordering of some topics may change, and we may spend more or less than one lecture per topic.