STAT 37710 / CMSC 35400, Spring 2020

Sign up for the waitlist here!

We will not be accepting auditors this quarter, but course lectures will be posted on this page as they become available.

This course provides a systematic view of a range of contemporary machine learning algorithms, as well as an introduction to the theoretical aspects of the subject. Topics covered include the statistical learning framework, estimation theory, model complexity, ensemble methods, mixture models, multilayer neural networks and deep learning, nonparametric methods, and active learning.
Prerequisites: Appropriate for graduate students who have taken Statistics 27700 & CMSC 25300/35300 (Mathematical Foundations of Machine Learning) or equivalent.


Logistics

  • Format: Pre-recorded lectures + live Zoom discussions during class time and office hours;
  • Instructor
  • Class hours: M/W, 1:30-2:50pm via Zoom. Please retrieve the Zoom meeting links on Canvas.
  • Office hours:

    • Tuesday 10-11am (Yuming Chen)
    • Thursday 9-10am (Lang Yu)
    • Friday 9-10am (Lang Yu)
    • Friday 2-3pm (Yuming Chen)
    • Saturday 9-10am (Roxie) — this is a “concept review” office hour focused on lecture and reading materials, NOT homework questions
    • Sunday 10-11am (Roxie) — this is a “concept review” office hour focused on lecture and reading materials, NOT homework questions
  • First lecture: 4/6/2020
  • Course website: https://voices.uchicago.edu/machinelearning/stats37710-cmsc35400-s20/
  • Canvas (announcements, homework assignments): https://canvas.uchicago.edu/courses/26960
  • Piazza (class discussion): Link to the Piazza page is provided on Canvas. The system is highly catered to getting you help quickly and efficiently from classmates, the TAs, and the instructors. Rather than emailing questions to the teaching staff, we encourage you to post your questions on Piazza. If you have any problems or feedback for the developers, email team@piazza.com.
  • Email policy: We will prioritize answering questions posted to Piazza, not individual emails.

Course materials

  1. Class Logistics Video, Notes (updated April 6)
  2. Probability and Statistics Review Video, Lecture Notes
  3. Statistical Learning Framework Video, Lecture Notes
  4. Bias-Variance Tradeoff Video, Lecture Notes, Bias-Variance Tradeoff Example Code, Makesig function
  5. Method of Moments Video, Lecture Notes
  6. MLE Video, Lecture Notes
  7. Bayesian Methods Part 1 Video, Part 2 Video, Lecture Notes, Deblurring Demo (Matlab)
  8. Complexity Regularization Video, Lecture Notes
  9. Decision Trees Video, Lecture Notes
  10. Logistic Regression Video, Lecture Notes
  11. Generative Classification Video, Lecture Notes
  12. Bagging video, Lecture Notes (Quiz on May 4 — may the fourth be with you!)
  13. Boosting video, Lecture Notes (Quiz on May 6)
  14. Gaussian Mixture Model Video, Lecture Notes (Quiz on May 11)
  15. Graphical Models Video, Lecture Notes (Quiz on May 13)
  16. Neural Networks, Part I, Video, Lecture Notes (Quiz on May 18)
  17. Neural Networks, Part II, Video, Lecture Notes (Quiz on May 20)
  18. Nonparametric Learning Part 1 Video (Kernel Ridge Regression), Lecture Notes (Quiz on May 27 — no class on Memorial Day)
  19. Nonparametric Learning Part 2 Video (Gaussian Processes), Lecture Notes (Quiz on May 27)
  20. Support Vector Machines video, Lecture Notes (Quiz on June 1)
  21. Active Learning Video, Lecture Notes (Quiz on June 3)

Tentative Schedule

  • Week 1: The statistical learning framework, bias-variance tradeoffs
  • Week 2: MLE, MAP, moment-based methods
  • Week 3: Model complexity (MDL, L0, L1, CART)
  • Week 4: Naive Bayes, LDA, logistic regression
  • Week 5: Ensemble methods: bagging, random forests, boosting
  • Week 6: Mixture models: mixtures of Gaussians, graphical models
  • Week 7: Multi-layer perceptrons and neural networks
  • Week 8: Nonparametric models: Kernel ridge regression, Gaussian processes
  • Week 9: Support vector machines; Active learning

Grading

All students will be evaluated through regular homework assignments and quizzes. The final grade will be allocated to the different components as follows:

  • 70%: Homework
    • There are roughly weekly homework assignments (about 8 total). Homework problems include both mathematical derivations and proofs as well as more applied problems that involve writing code and working with real or synthetic data sets.
  • 30%: Quizzes
    • Quizzes will be via canvas and cover material from the assigned videos.

Letter grades will be assigned using the following hard cutoffs:

  • A: 93% or higher
  • A-: 90% or higher
  • B+: 87% or higher
  • B: 83% or higher
  • B-: 80% or higher
  • C+: 77% or higher
  • C: 60% or higher
  • D: 50% or higher
  • F: less than 50%

We reserve the right to curve the grades, but only in a fashion that would improve the grade earned by the stated rubric.

  • Late Policy. Late homework and quiz submissions will lose 5% of the available points per day late.
  • Pass/Fail Grading.¬†A grade of P is given only for work of C- quality or higher. You should make the request for Pass/Fail grading in writing (private note on Piazza). You must request Pass/Fail grading¬†prior to the last day of class (by June 2).

Zoom Policies and Expectations

Spring Quarter 2020 Recording Policy

As the University temporarily transitions to a remote teaching and learning environment, instructors
and students have asked for guidance on the recording of course sessions. Instructors have the
discretion to record course sessions, except when recording is required to meet the needs of students
granted an accommodation by the Office of Student Disability Services. Recordings and transcripts will
be made available to students in the relevant course, the instructor, and other necessary University
officials. Recordings in which students are personally identifiable will be managed in accordance with
the Family Educational Rights and Privacy Act (FERPA).

This time-limited policy has been implemented to effectively deliver a remote education while
safeguarding privacy and protecting rights in courses and instructional materials. Below is an
acknowledgment for students designed to govern the use of any recordings and provide instructors and
students with guidance on the use of instructional materials.

By attending course sessions, students acknowledge that:

  1. They will not: (i) record, share, or disseminate University of Chicago course sessions, videos,
    transcripts, audio, or chats; (ii) retain such materials after the end of the course; or (iii) use such
    materials for any purpose other than in connection with participation in the course.
  2. They will not share links to University of Chicago course sessions with any persons not
    authorized to be in the course session. Sharing course materials with persons authorized to be
    in the relevant course is permitted. Syllabi, handouts, slides, and other documents may be
    shared at the discretion of the instructor.
  3. Course recordings, content, and materials may be covered by copyrights held by the University,
    the instructor, or third parties. Any unauthorized use of such recordings or course materials may
    violate such copyrights.
  4. Any violation of this policy will be referred to the Area Dean of Students.

Textbooks

FAQs

  • Do you allow auditors?
    • No. Unfortunately we can not accommodate such requests due to high demand.
  • What if I cannot join the class hours due to unavoidable conflicts?
    • The course materials are available on the course website. You can interact with us during the live hours and via Piazza.
  • What’s the best way to interact with the instructors and TAs?
    • Besides the live class and office hours, the best way to reach out to us offline is via Piazza. Before posting your question, please go through the questions posted and make sure that your question has not already been answered.
  • How do I participate in the class and office hours?
    • For live discussions during class time and office hours, find the Zoom link for the corresponding session on Canvas. You may be prompted to log in with your UChicago ID and password.
  • How do I access the homework and quizzes?
    • HWs and quizzes are not publically available, and can only be accessed through Canvas for students enrolled in this class.