Instructor: Furong Huang

When and where: Tuesday/Thursday 3:30-4:45pm PLS 1140

Office hours: Thursday 4:45-5:45pm

TA: Anton Jeran Ratnarajah (Office hours: Monday 6:00-7:00pm),

Wichayaporn Wongkamjan (Office hours: Friday 4:00-5:00pm)

Piazza: CMSC742 Piazza

Our primary source of readings will be Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning. MIT Press, 2012. We will also read papers and learn materials that are not yet in textbooks.

Other recommended (but not required) books:

- Machine Learning: The Art and Science of Algorithms that Make Sense of Data by Peter Flach (ISBN 1107422221)
- Pattern Recognition and Machine Learning by Chris Bishop (ISBN 0387310738)
- Machine Learning by Tom Mitchell (ISBN 0070428077)
- Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman (ISBN 0387952845)
- Information Theory, Inference and Learning Algorithms by David MacKay (ISBN 0521642981)
- An Introduction to Computational Learning Theory by Michael Kearns and Umesh Vazirani (ISBN 0262111934)

For RL, here are some good books that you can consult:

- Markov Decision Processes: Discrete Stochastic Dynamic Programming, by Martin Puterman.
- Reinforcement Learning: An Introduction, by Rich Sutton and Andrew Barto. (draft available online)
- Algorithms of Reinforcement Learning, by Csaba Szepesvari. (pdf available online)
- Neuro-Dynamic Programming, by Dimitri Bertsekas and John Tsitsiklis.

Papers to be discussed will be made available to ahead of time.

Useful inequalities cheat sheet (by László Kozma)

Concentration of measure (by John Lafferty, Han Liu, and Larry Wasserman)

Machine learning studies automatic methods for learning to make accurate predictions, to understand patterns in observed features and to make useful decisions based on past observations.

This course introduces theoretical machine learning, including mathematical models of machine learning, and the design and rigorous analysis of learning algorithms.

Learning theory (traditional and modern)

PAC learning basics

Boosting theory

PAC learning in neural nets

Latent variable graphical models

Graphical model basics

Spectral methods: matrix/tensor decomposition

Reinforcement learning theory

RL overview: algorithms and analyses

RL theory: sample complexity

Minimum grade of A- in CMSC422 (or equivalent) or CMSC498M; permission of CMNS-Computer Science department and instructor.

- Basic machine learning concepts
- Supervised/unsupervised/reinforcement learning
- Classification, Regression, Cross validation, Overfitting, Generalization
- Deep neural networks
- See Math for Machine Learning

- Basic calculus and linear algebra
- Compute (by hand) gradients of multivariate functions
- Conceptualize dot products and matrix multiplications as projections
- Solve multivariate equations using, etc, matrix inversion, etc
- Understand basic matrix factorization
- See linear algebra review, and advanced

- Basic optimization
- Use techniques of Lagrange multipliers for constrained optimization problems
- Understand and be able to use convexity
- See convex analysis review, optimization review

- Basic probability and statistics
- Understand: random variables, expectations and variance
- Use chain rule, marginalization rule and Bayes' rule
- Make use of conditional independence, and understand "explaining away"
- Compute maximum likelihood solutions for Bernoulli and Gaussian distributions
- See probability review

The purpose of assignments & grading is to provide extra incentive to help you keep up with the material and assess how well you understand it, so that you have a solid background in machine learning by the end of the semester.

I expect students to

- Come to class prepared, having completed the assigned readings.
- Complete the assigned weekly homework assignments before class, and be prepared to discuss their solution in class.
- Participate actively in discussions both in person and online.

Your grade will be based on: (the following rubric can be updated after semester starts)

**Homeworks (20%).**There will be roughly one homework per week released on ELMS. Each is worth 1 to 2% of your final grade, depending on its length. The weights will be marked accordingly. Homeworks that are not autograded will be graded on a high-pass (100%), low-pass (50%) or fail (0%) basis. These are to be completed individually.**Course projects (30%).**These must be completed in teams of 3-5 students. You will submit a proposal, a progress report and a final report, as well as present your project.**Lecture scribing (40%).**Sign Up ASAP. Each student scribes at least 1 session. Students are responsible to provide comments for all scribed notes. Grading based on quality of submitted notes (25%) and feedback provided for other notes (15%).**Participation (10%).**(1) Submit feedback about last session at the beginning of each session on Piazza. (2) Ask and answer questions on piazza. Grading based on activity level on Piazza.

President Pines provided clear expectations to the University about the wearing of masks for students, faculty, and staff. Face coverings over the nose and mouth are required while you are indoors at all times. There are no exceptions when it comes to classrooms, laboratories, and campus offices. Students not wearing a mask will be given a warning and asked to wear one, or will be asked to leave the room immediately. Students who have additional issues with the mask expectation after a first warning will be referred to the Office of Student Conduct for failure to comply with a directive of University officials.

Late homeworks are not allowed. Period. No exceptions. The time deadlines are automatic and unforgiving.

Late projects are allowed: you get two extra days. However, once the project is 1 minute late, you lose 25% (absolute).

If you handed something in and do not get a score for an assignment, you have one week to let us know about the issue.

Any student who needs to be excused for an absence from a single lecture, recitation, or lab due to a medically necessitated absence shall make a reasonable attempt to inform the instructor of his/her illness prior to the class. Upon returning to the class, present their instructor with a self-signed note attesting to the date of their illness. Each note must contain an acknowledgment by the student that the information provided is true and correct. Providing false information to University officials is prohibited under Part 9(i) of the Code of Student Conduct (V-1.00(B) University of Maryland Code of Student Conduct) and may result in disciplinary action.

Self-documentation may not be used for the Major Scheduled Grading Events you define in your syllabus (e.g., midterm exams, project presentations, etc.) and it may only be used for only 1 class meeting (or more, if you choose) during the semester. Any student who needs to be excused for a prolonged absence (2 or more consecutive class meetings), or for a Major Scheduled Grading Event, must provide written documentation of the illness from the Health Center or from an outside health care provider. This documentation must verify dates of treatment and indicate the timeframe that the student was unable to meet academic responsibilities. In addition, it must contain the name and phone number of the medical service provider to be used if verification is needed. No diagnostic information will ever be requested.

Please document **in writing** (e.g., email) the accommodations you are making for a student who missed a graded assignment. Common accommodations include allowing the student to retake an exam or submit an assignment late, or changing the grading scheme to account for ungraded assignments.

Any student eligible for and requesting reasonable academic accommodations due to a disability is requested to provide, to the instructor in office hours, a letter of accommodation from the Office of Disability Support Services (DSS) **within the first TWO weeks of the semester.**

In this course you are responsible for both the University’s Code of Academic Integrity and the University of Maryland Guidelines for Acceptable Use of Computing Resources. Any evidence of unacceptable use of computer accounts or unauthorized cooperation on tests and assignments will be submitted to the Student Honor Council, which could result in an XF for the course, suspension, or expulsion from the University.

**Note that posting project solutions in a public online location is a violation of your academic integrity policy.**

Any homework or exam that is handed in must be your own work. However, talking with one another to understand the material better is strongly encouraged. Recognizing the distinction between cheating and cooperation is very important. If you copy someone else's solution, you are cheating. If you let someone else copy your solution, you are cheating. If someone dictates a solution to you, you are cheating. Everything you hand in must be in your own words, and based on your own understanding of the solution. If someone helps you understand the problem during a high-level discussion, you are not cheating. We strongly encourage students to help one another understand the material presented in class, in the book, and general issues relevant to the assignments. When taking an exam, you must work independently. Any collaboration during an exam will be considered cheating. Any student who is caught cheating will be given an E in the course and referred to the University Student Behavior Committee. Please don't take that chance - if you're having trouble understanding the material, please let us know and we will be more than happy to help.

The open exchange of ideas, the freedom of thought and expression, and respectful scientific debate are central to the aims and goals of this course. These require a community and an environment that recognizes the inherent worth of every person and group, that fosters dignity, understanding, and mutual respect, and that embraces diversity. Harassment and hostile behavior are unwelcome in any part of this course. This includes: speech or behavior that intimidates, creates discomfort, or interferes with a person’s participation or opportunity for participation in the course. We aim for this course to be an environment where harassment in any form does not happen, including but not limited to: harassment based on race, gender, religion, age, color, national origin, ancestry, disability, sexual orientation, or gender identity. Harassment includes degrading verbal comments, deliberate intimidation, stalking, harassing photography or recording, inappropriate physical contact, and unwelcome sexual attention. Please contact an instructor or CS staff member if you have questions or if you feel you are the victim of harassment (or otherwise witness harassment of others).

We welcome your suggestions for improving this class, please don’t hesitate to share it with the instructor or the TA during the semester! You will also be asked to give feedback using the CourseEvalUM system at the end of the semester. Your feedback will help us make the course better.

Although every effort has been made to be complete and accurate, unforeseen circumstances arising during the semester could require the adjustment of any material given here. Consequently, given due notice to students, the instructor reserves the right to change any information on this syllabus or in other course materials.