CMSC 818J - Domain Specific Architectures (DSA)

Hello! Welcome to the Spring 2023 edition of CMSC 818J at UMD CS!

Administrative Information

Description and topics

Domain-Specific Architectures (DSAs):

Today, most of the computations in the world, from large-scale data center applications to smaller programs on home appliances, are mainly running on general-purpose programmable CPUs, and more recently on GPUs, too. While programmability has made CPUs popular and easy to deploy in a wide range of application domains, it has made computation less power/cost-efficient and less scalable because even to perform the simplest logical operation (e.g., AND, OR) at hardware, several sophisticated steps are required to translate a software code to the hardware operation. For many years, while Moore’s law was still valid and the number of transistors on a microchip could double every two years, the costs associated with CPU-based computations had been acceptable. However, as Moore’s law has been dying (at least for certain technologies such as on-chip memory), to keep scaling performance efficiently, we must design domain-specific architectures (DSAs) that utilize a given hardware budget more efficiently by focusing on the implantation of only the operations that a particular application requires rather than supporting programmability. DSAs (a.k.a., hardware accelerators), have been proposed/designed for several applications with distinct characteristics at various scales including deep learning, computer vision, scientific computing, database systems, etc. Besides the growing research in academia, the industry has been heavily investing in developing DSAs. These days, any device you buy (e.g., cell phones or laptops) definitely includes at least one DSA. Just a couple of examples of many DSAs in today’s industry are Google’s TPU and Apple’s Neural Engine.

Course Objectives:

This course explores recent advances in both academia and industry in designing DSAs. This course overviews the characteristics of applications that enable them to greatly benefit from DSAs and the source of speedup in DSAs which mainly includes parallelism, memory access optimization, and data specialization.

Learning Outcomes:

As part of this course, students will get familiar with common techniques such as using systolic arrays and near-data processing in state-of-the-art DSAs to enable efficient computation. The course also summarizes the concept of hardware-software co-design. This course helps students understand key design-space trade-offs (e.g., balancing area-power-performance and/or specialization-generality) to design efficient DSAs and highlights important benchmarking metrics to evaluate the rapidly growing number of DSAs being developed in academia and industry.

Course Structure and Content:

This course includes lectures interspersed with heavy paper reading and discussions. The material for this course will be derived from recent papers from major computer architecture conferences (i.e., ISCA, MICRO, HPCA, ASPLOS) on DSAs. The semester-long assignments of this course will include paper reading/critique plus coding practices for DSA implementations or studies. The course also includes a research-oriented project that allows students to explore outstanding research topics in this field and propose their own solutions. To this end, each student first explores different application domains to choose one to work on. Then, they will be guided to take the following steps to proceed with their projects (each step has a partial credit). The selected final reports are encouraged to submit their work as a research paper and will also be supported to present their work at conferences.

Topics:

This course will cover topics including:

Prerequisites

Communication and discussion forums

For communication and sharing resources we will be using:

Material and textbooks

This course is mainly based on recent papers from major computer architecture conferences

Evaluation

Tentative schedule

Week Date Lesson Released Due Comments
1 01/25/2023 Logistics + Introduction to DSAs Assignments 0, 1
2 01/30/2023 Dataflow I Assignments 0 Roofline model, pipelining, and systolic arrays
2 02/01/2023 Dataflow II + Streaming I Instructions for topic exploration (form) Systolic Arrays for DNNS, Streaming, Little's law, and decompression.
3 02/06/2023 Streaming II The impact of data dependency challenges on streaming
3 02/08/2023 HW/SW Co-design Assignment 1 hw/sw co-design for efficient streaming, rethinking systolic arrays for solving PDEs
4 02/13/2023 Near-Data Processing (NDP) Assignment 2 Presentation phase I paper selection (optional form) We explore NDP solutions for recommendation systems
4 02/15/2023 Practice Questions for Midterm Instruction for literature reivew and paper critique. Presentation phase I schedule and critique assignments. Topic selection (form)
5 02/20/2023 TBD
5 02/22/2023 Midterm Date of midterm is subject to change
6 02/27/2023 DSAs for scientific computing Presentation phase II schedule
6 03/01/2023 Analong-digital and sparse-dense accelerators Assignment 2 (3/3)
7 03/06/2023 Sparse computation and compression I Instructions for proposal
7 03/08/2023 Sparse computation and compression II Literature review (3/10)
8 03/13/2023 Sparsity in neural networks plus systolic arrays I Instructions for final report
8 03/15/2023 Sparsity in neural networks plus systolic arrays II Proposal (3/17)
9 03/20/2023 No Class Spring Break
9 03/22/2023 No Class Spring Break
10 03/27/2023 Recommendation systems and near memory solutions I
10 03/29/2023 Recommendation systems and near memory solutions II
11 04/03/2023 Graph analytics Template for final presentation and instructions for peer review
11 04/05/2023 Deep dive into sparsity I Milestone I and experimental setup (4/7)
12 04/10/2023 Deep dive into sparsity II
12 04/12/2023 Neuroscience and genome processing
13 04/17/2023 Security and privacy
13 04/19/2023 Transformers
14 04/24/2023 Deep dive into HW/SW co-design and NDP
14 04/26/2023 TBD Milestone II (4/28)
15 05/01/2023 Final Presentations (topic:TBD))
15 05/03/2023 Final Presentations (topic:TBD)
16 05/08/2023 Final Presentations (topic:TBD)
16 05/10/2023 Final Presentations (topic:TBD) Final Report

Disabilities Support Accommodations

In case academic accommodations are needed, you must provide a letter of accommodation from the Office of Accessibility and Disability Services (ADS) within the first two weeks of the semester. For details, see the section titled “Accessibility” available at Course Related Policies.

Mask Policy

Please check the latest campus policies regarding COVID protocols: https://umd.edu/4Maryland. There has been a recent update in the mask mandate: “Effective Monday, August 29, 2022, wearing a mask will not be required while indoors in most situations, including classrooms. However, wearing a KN95 mask is recommended while indoors for added protection.”

Academic Integrity

Academic dishonesty includes not only cheating, fabrication, and plagiarism, but also includes helping other students commit acts of academic dishonesty by allowing them to obtain copies of your work. In short, all submitted work must be your own. Cases of academic dishonesty will be pursued to the fullest extent possible as stipulated by the Office of Student Conduct.

It is very important for you to be aware of the consequences of cheating, fabrication, facilitation, and plagiarism. If the student is found to be responsible for academic dishonesty, the typical sanction results in a special grade “XF”, indicating that the course failed due to academic dishonesty. If you have any doubt as to whether an act of yours might constitute academic dishonesty, please contact your TA or the course coordinator.

The CS Department takes academic integrity seriously. Information on how the CS Department views and handle academic integrity matters can be found at Academic Integrity.

A few examples of academic integrity violations

Additional information can be found in the sections titled “Academic Integrity” and “Code of Student Conduct” available at Course Related Policies.

For more information on the Code of Academic Integrity or the Office of Student Conduct, visit https://studentconduct.umd.edu.

Excused Absences

If you need to be excused for an absence from a single lecture due to a medical reason, you shall make a reasonable attempt to inform the instructor of your illness prior to the class. Upon returning to the class, you will present a self-signed note attesting to the date of your illness. Each note must contain an acknowledgment by the student that the information provided is true and correct. Providing false information to University officials is prohibited under Part 9(i) of the Code of Student Conduct (V-1.00(B) the University of Maryland Code of Student Conduct) and may result in disciplinary action.

Missing an exam for reasons such as illness, religious observance, participation in required university activities, or family or personal emergency (such as a serious automobile accident or close relative’s funeral) will be excused so long as the absence is requested in writing in advance and the student includes documentation that shows the absence qualifies as excused.

A self-signed note is not sufficient for exams because they are Major Scheduled Grading Events. In the case of medical absence, you must furnish documentation from the health care professional who treated you. The documentation must clearly include verification of (1) treatment dates and (2) the time period for which the student is unable to meet academic responsibilities. In addition, it must contain the name and phone number of the medical service provider to be used if verification is needed. No diagnostic information will ever be requested. Note that simply being seen by a health care professional does not constitute an excused absence; it must be clear that you were unable to perform your academic duties.

For additional details, see the section titled “Attendance and Missed Assignments” available at Course Related Policies.

Copyright

All course materials are copyright UMCP, Department of Computer Science © 2022. All rights reserved. Students are permitted to use course materials for their own personal use only. Course materials may not be distributed publicly or provided to others (except other students in the course), in any way or format.

Although every effort has been made to be complete and accurate, unforeseen circumstances arising during the semester could require the adjustment of any material given here. Consequently, given due notice to students, the instructor reserves the right to change any information on this syllabus or in other course materials.

Concerns

If you have any class concerns, please feel free to contact the instructor. If an issue arises with the instructor, report it using the form available at https://www.cs.umd.edu/classconcern.

UMD Web Accessibility