EECS 440: Machine Learning (Fall 2009)

Instructor: Soumya Ray
- Office: Olin 516
- Office Hours: 3:30-5:00pm Monday, or by appointment

Meeting Time and Location: MW 2-3:15, Sears 354

Textbook: Machine Learning by Tom M. Mitchell

Books at KSL (2 hour reserve):
- Pattern Classification (Duda, Hart and Stork)
- Pattern Recognition and Machine Learning (Bishop)
- Machine Learning (Mitchell)

News

8/3 Course website is up.
8/26 Programming Assignment 0 is assigned.
8/27 Written Homework 1 is online, due September 16.
9/2 Programming Assignment 1 is online, due Sept 28.
9/21 Written Homework 2 is online, due October 12.
9/28 Programming Assignment 2 is online, due Oct 21.
10/19 Written Homework 3 is online, due Nov 11.
10/19 Think about course projects. Send one paragraph summaries by Friday 23. I will meet with everyone next week.
11/18 Written Homework 3 is online, due Dec 9.

Overview

This course is focused on algorithms for machine learning: their design, analysis and implementation.
We will study different learning settings, including supervised, semi-supervised and unsupervised learning.
We will study different ways of representing the learning problem, using propositional, multiple-instance and relational representations.
We will study the different algorithms that have been developed for these settings, such as decision trees,
neural networks, support vector machines, k-means, harmonic functions and Bayesian methods. We will learn about the
theoretical tradeoffs in the design of these algorithms, and how to evaluate their behavior in practice.
At the end of the course, you should be able to:

Recognize situations where machine learning algorithms are applicable
Understand, represent and formulate the learning problem
Apply the appropriate algorithm(s), or if necessary, design your own, with an understanding of the tradeoffs involved
Correctly evaluate the behavior of the algorithm when solving the problem.

Grading

Homework: 30%
Programming Assignments: 20%
Project: 30%
Exam: 15%
Class Participation: 5%

Assignments are due by 5 p.m. on the due date. Assignments submitted late are penalized @ 10% for each late day. If you have a very good reason, such as illness, please see me.
Collaboration policy: You are welcome to discuss assignments with each other and with me, but all your submitted work must be your own.
Case Academic Integrity Policy

Lecture Notes

Date	Topic	Textbook Chapter/Reading	Lecture Notes
Aug 24	Background 1		Lecture 1
Aug 26	Background 2		Lecture 2
Aug 31	Foundations/ Propositional Supervised Learning	Chapter 1 and 2	Lecture 3
Sep 2	Decision Tree Induction	Chapter 3	Lecture 4
Sep 7	No class	-----	-----
Sep 9	Decision Tree Induction	Chapter 3	Lecture 5
Sep 14	Evaluation Methodology and Metrics		Lecture 6
Sep 16	Artificial Neural Networks	Chapter 4	Lecture 7
Sep 21	Artificial Neural Networks	Chapter 4	Lecture 8
Sep 23	Comparing Learning Algorithms, Hypothesis Testing	Chapter 5	Lecture 9
Sep 28	Hypothesis Testing	Chapter 5	(See previous lecture)
Sep 30	Support Vector Machines		Lecture 10
Oct 5	Support Vector Machines		SVM Tutorial
Oct 7	Bayesian Learning	Chapter 6	Lecture 11
Oct 7	Naive Bayes and Logistic Regression	Chapter 6	Lecture 12
Oct 12	Naive Bayes and Logistic Regression	Chapter 6	(See previous lecture)
Oct 14	Midterm Exam	-----	-----
Oct 19	No class	-----	-----
Oct 21	Mixture models and Expectation Maximization	Chapter 6	Lecture 13
Oct 21	Bias-Variance Analysis		(See Lecture 13)
Oct 26	Computational Learning Theory	Chapter 7
Oct 28	Computational Learning Theory	Chapter 7	Lecture 14
Nov 2	Learning with Prior Knowledge
Nov 4	Learning with Prior Knowledge		Lecture 15
Nov 9	Ensemble Methods		Lecture 16
Nov 11	Bayesian Networks	Chapter 6.11	Lecture 17
Nov 16	Inference in Bayesian Networks		see previous lecture, Junction Tree notes
Nov 18	Parameter and Structure Estimation in BNs		(see previous lecture)
Nov 23	Multiple-Instance and Relational Learning	Chapter 10
Nov 25	Sequential Supervised Learning
Nov 30	Unsupervised Learning
Dec 2	Wrapup
Dec 9	Final Project Presentation
Dec 14	Final Project Presentation

Assignments

All dates below are tentative until the assignments are actually posted.

Assignment	Date Assigned	Date Due	Data/Code
Written 1	Aug 31	Sep 16
Written 2	Sep 21	Oct 12
Written 3	Oct 21	Nov 11	Matlab Script for Question 8(b)
Written 4	Nov 18	Dec 9
Programming 0	Aug 26	Do not turn in	example.zip
Programming 1	Sep 2	Sep 28	Data
Programming 2	Sep 28	Oct 21
Course Project	Oct 26	Dec 14

Fun Stuff

The Discipline and Future of Machine Learning. Video of a seminar by Tom Mitchell, one of the founders of the field.
Bayesian models of human inductive learning. Video of invited talk by Josh Tenenbaum, ICML 2007.

Last update November 18, 2009 5:35 PM by Soumya Ray