Eric Siegel now provides predictive analytics services at Prediction Impact, Inc., where predictive modeling and data mining are applied to gain customer intelligence.

Welcome to Machine Learning. This course covers technology that allows computers to learn from example, i.e., supervised learning.

In one view of machine learning, the computer searches for a "meta-solution" to a problem, instead of just a solution. That is, instead of solving one instance of the problem, it derives an hypothesis that describes how to solve the problem in general. Think of it this way: you can search for the right words to express a thought, or you can search for the best writing style. Likewise, instead of searching for the best place to drop your next piece when playing the game Tetris, search for an algorithm to do so more efficiently.

In this course, you will:

• survey various forms, such as decision trees, to describe such learned hypotheses, and various search methods that create them.
• learn the fundamentals and the latest developments in machine learning
• gain both theoretical knowledge of and hands-on experience with machine learning techniques
• survey the increasingly broad range of machine learning applications, from protein folding to data mining

Learn This! -- a rap about machine learning.

Homework Assignments: (subject to modification)

• HW 1: Basic ML Concepts
• HW 2: Decision Trees for Credit and Medical
• HW 3: Neural Networks for Face Recognition
• HW 4: Genetic Programming for Othello
• HW 5: Final Project

Select visuals from lectures: (subject to modification)

• Lecture 1
• Lecture 3
• Lecture 10 (understanding NN backprop)
• Lecture 11, and a little 12 (GAs)
• Lecture 13 (GP and ADFs)
• Lecture 14 (Midterm review)
• Lecture 21 (meta-learning, co-evolution, competition)
• Lecture 22
• Lecture 24 (summary)

Learning Methods Covered:

• Version spaces
• Decision trees
• Neural networks
• Bayesian learning
• Genetic algorithms
• Genetic programming
• Hybrid learning

Learning Topics Discussed:

• Applications: from robots to natural language
• Representation power: from weights to programs
• Representation issues: modularity, ontogeny
• Overfitting the data
• Learning issues: search strategy, credit assignment
• Evaluating results
• Meta-learning helps learning
• Cooperative and competitive coevolution

Information

Text: Machine Learning, by Tom M. Mitchell, available at Papyrus Books, west side of Broadway a couple blocks down from 116th.

Grading : 65% homeworks, 15% midterm, 20% final. There will be about six homework assignments, four involving programming projects and all involving learning theory.

Late homework: Late assignments will be penalized. Note that partial credit will be considered for all incomplete work.

Collaboration: Discussion of material covered in class is strongly encouraged. Some assignments will be designated as collaborative. Otherwise, the work you submit must be your own work. There is a line between discussion and cheating and this line will be strictly enforced.

Open Door Policy: We would like the course to run smoothly and enjoyably. Feel free to let us know what you find good and interesting about the course. Let us know sooner about the reverse. See us, leave us a note, or send us an e-mail.

email: evs at cs dot columbia dot edu