CMSC 828W
Home
Course info
Resources

CMSC 828W: Foundations of Deep Learning

University of Maryland, College Park, Fall 2020
Instructor: Soheil Feizi

Instructor

Soheil Feizi, sfeizi@cs.umd.edu
Office: 4202 IRB Building

Course Assistants

Aya Abdelsalam Ismail, asalam@cs.umd.edu

Neha Kalibhat, nehamk@cs.umd.edu

Mucong Ding, mcding@terpmail.umd.edu

Topics

We will attempt to cover the following topics

  • ERM and basic DL models/architectures

  • DL Optimization

  • DL Generalization

  • Neural Tangent Kernels

  • Lottery Tickets

  • Robustness

    • Adversarial attacks and defenses

    • Provable defenses, Lp and non-Lp

    • Generalizable robustness to unforeseen attacks & poisoning attacks

  • Relaxing i.i.d.

    • Domain adaptation

    • Out-of-domain generalization

  • Deep Generative Models

    • VAEs

    • GANs

    • Flow-based models

  • Min-Max Optimization

  • Contrastive, Self-supervised DL

  • Deep Reinforcement Learning

  • LSTMs and Transformers

  • Bayesian DL

  • Explainable DL

    • Interpretation methods

    • Influence functions

  • Fairness

  • Graph Neural Networks

  • Meta Learning

  • Federated DL

  • Privacy and Ethics

Note that this is a tentative list, and we may add or remove some topics as it fits.

Lectures

Lectures are on Tuesdays and Thursdays, 11:00 am – 12:15 pm, on Zoom.

Students are strongly encouraged to attend all the lectures. Relevant notes will be posted a couple of hours after each lecture.

Course Piazza:

Register here.

Piazza will be used for the review process of the mini-conference. Make sure to register if you are enrolled.

Office Hours

The instructor and the course assistant will provide weekly office hours.

  • Soheil Feizi: Tuesdays 12:15 pm – 1:15 pm.

  • TA office hours: TBD.

Prerequisites

CMSC 828W is a mathematical course. Strong linear algebra, probability, and optimization background are required. Students should be familiar with basic machine learning and deep learning concepts.

We will make extensive use of the Python programming language. It is assumed that you know or will quickly learn how the program in Python. You can satisfy this requirement by completing a programming course that uses Python and object-oriented techniques.

Course requirements and grading

Mini-Conference

  • CMSC 828W is a project-oriented course. Projects should be based on new works, cannot be resubmissions of previous research works by the team.

  • The format of the mini-conference is double-blinded:

    • Authors should not reveal their names in the submissions.

    • Each student would review 3 submissions. Reviewers should not reveal which papers are reviewing for. Any violation would cause a 25% penalty of the associated scores.

    • Formating violations will cause up to 25% penalty of the associated paper score.

    • Each submission should have up to 4 students.

    • Deadlines are final and won't be extended.

  • Important Dates

    • Topic and team formation deadline: 10/1

    • Topic feedback: 10/7

    • Abstract submission deadline: 11/5

    • Full paper (ICML format) submission deadline: 11/13

    • Review period: 11/14 – 11/20

    • Rebuttal period: 11/20 – 12/1

    • Review discussions: 12/1 – 12/4

    • AC final decisions and evaluating reviews: 12/4 – 12/15

Lecture Scribing

  • The scribed notes are on Overleaf: Click here (view only).

  • Each student is responsible for scribing one lecture in LaTeX format on Overleaf. In general, 3 – 4 students will be assigned to one lecture. They should work together in scribing the lecture. The scribe should elaborate on parts of the lecture that have been merely skimmed over, have full mathematical details, and provide proper references.

  • Sign up for the lecture you want to scribe using this google sheet.

  • You will be completing the lecture scribe on Overleaf. The link will be posted to the corresponding scribers via Piazza shortly after each lecture.

  • The Tuesday lecture scribe will be due on Saturday in the same week by 6:00 pm, and the Thursday lecture scribe will be due on the following Monday by 6:00 pm.

  • After completing your part of the scribe, please add a summary of the contributions you made to the google sheet under the column “contributions.”

Exams

  • We will have one final take-home exam on 12/10.

Grading

  • Lecture Scribe 10%

  • Final exam 25%

  • Paper score 40%

  • Review score 25%

Accommodations and Policies

You can find UMD's course policies here.

Any student eligible for and requesting reasonable academic accommodations due to a disability is requested to provide, to the instructor in office hours, a letter of accommodation from the Office of Accessibility and Disability Services (ADS, formerly DSS) within the first TWO weeks of the semester.

Any student who needs to be excused for an absence from a single lecture, recitation, or lab due to a medically necessitated absence shall:

  • Make a reasonable attempt to inform the instructor of his/her illness prior to the class.

  • Upon returning to the class, present their instructor with a self-signed note attesting to the date of their illness. Each note must contain an acknowledgment by the student that the information provided is true and correct. Providing false information to University officials is prohibited under Part 10(j) of the Code of Student Conduct (V-1.00(B) University of Maryland Code of Student Conduct) and may result in disciplinary action.

If you observe religious holidays during the course, please notify course staff within the first two weeks of the semester.

Course Evaluations

Course evaluations are important and that the department and faculty take student feedback seriously. Students can go to the link to complete their evaluations.