EECS 600--103: Machine Learning (Fall 2008)
- Instructor: Soumya Ray
- Office: Olin 516
- Office Hours: 3-4:30pm Friday, or by appointment
- Meeting Time and Location: MW 2-3:15, Nord 410
(we will meet Fridays if needed)
- Textbook: Machine Learning by Tom M. Mitchell
- Books at KSL (2 hour reserve):
- Pattern Classification (Duda, Hart and Stork)
- Pattern Recognition and Machine Learning (Bishop)
- Machine Learning (Mitchell)
News
8/18 Course website is up.
8/28 Homework 1 is assigned, due September 15.
9/5 Programming Assignment 1 is online, due September 29. Deadline extended to October 6
9/23 Homework 2 is online, due October 13. Deadline extended to October 15
9/30 Programming Assignment 2 is online, due October 22. Deadline extended to October 29
10/7-10/21 Class Project meetings.
10/22 Midterm exam, Nord 410, 2-3:15. Class Projects are assigned. Class Presentations on December 8 and 10. Writeup due December 15.
11/7 Homework 3 is assigned, due November 24.
11/26 Homework 4 is assigned, due December 15.
12/18 Grades assigned. Good job everyone!
Overview
This course is focused on algorithms for machine learning: their design, analysis and implementation.
We will study different learning settings, including supervised, semi-supervised and unsupervised learning.
We will study different ways of representing the learning problem, using propositional, multiple-instance and relational representations.
We will study the different algorithms that have been developed for these settings, such as decision trees,
neural networks, support vector machines, k-means, harmonic functions and Bayesian methods. We will learn about the
theoretical tradeoffs in the design of these algorithms, and how to evaluate their behavior in practice.
At the end of the course, you should be able to:
- Recognize situations where machine learning algorithms are applicable
- Understand, represent and formulate the learning problem
- Apply the appropriate algorithm(s), or if necessary, design your own, with an understanding of the tradeoffs involved
- Correctly evaluate the behavior of the algorithm when solving the problem.
Grading
Homework: 20%
Programming Assignments: 20%
Project: 30%
Exam: 20%
Class Participation: 10%
Lecture Notes
Introduction
Lecture 2: Foundations, kinds of learning problems, example representation (Read
Mitchell Chapters 1 & 2)
Lecture 3: Propositional Supervised Learning, Decision Tree Induction (Read Mitchell
Chapter 3)
Lecture 4: Decision Tree Induction, Overfitting (Solve problem on slide 17 for Friday)
Lecture 5: Overfitting
Lecture 6: Evaluation Methodology and Metrics
Lecture 7: Artificial Neural networks (Read Mitchell Chapter 4)
Lecture 8: Artificial Neural networks
Lecture 9: Comparing Learning Algorithms and hypothesis testing (Read Mitchell Chapter 5)
Lecture 10: Support Vector Machines
Lecture 11: Support Vector Machines Optional reading
Lecture 12: Bayesian Learning (Read Mitchell Chapter 6)
Lecture 13: Naive Bayes, Logistic regression, Mixture models, EM
Lecture 14: Bias-Variance Analysis
Lecture 15: Computational Learning Theory (Read Mitchell Chapter 7)
Lecture 16: Learning with Prior Knowledge (Read Mitchell Chapter 12)
Lecture 17: Learning with Prior Knowledge
Lecture 18: Midterm Exam, Oct 22
Lecture 19: Ensemble Methods (Extra Class Friday Oct 24) Survey of
ensemble methods Tutorial on Boosting Gradient Boosting
Lecture 20: Bayesian networks (Read Mitchell 6.11) (Optional Reading: "AI: A modern approach", Russell & Norvig, Chapter 14)
Lecture 21: Inference in Bayesian Networks
Lecture 22: Parameter and Structure Estimation for Bayesian Networks
Lecture 23: Multiple-Instance Learning
Lecture 24: Relational Learning (Read Mitchell Chapter 10)
Lecture 25: Relational Learning
Lecture 26/27: Sequential Supervised Learning (Optional Reading: "AI: A modern approach", Russell & Norvig,
Chapter 15)
Lecture 28: Machine learning in Bioinformatics, guest lecture by Prof. Mehmet Koyuturk
Lecture 29: Machine learning in Software Engineering, guest lecture by Prof. Andy Podgurski
Lecture 30: Unsupervised learning Optional Reading Spectral Clustering
Lecture 31: Class project presentations
Lecture 32: Class project presentations
Lecture 33 (extra class): Wrapup
Assignments
Homework 1, due Monday September 15
Programming Assignment 1, due Monday September 29 (October 6). Example Problem
Test problem
Homework 2, due Monday October 13 ( October 15).
Programming Assignment 2, due Wednesday October 22 ( October 29). Use the example and test problems above to
debug/compare your
results with others. You can also use the problems we reviewed in class (see Lecture 14).
Homework 3, due Monday November 24. Matlab script
Homework 4, due Monday December 15.
Fun Stuff
Last update December 18, 2008 4:40 PM by Soumya Ray