Readings
CMSC 714:  High Performance Computing

CMSC 714
Syllabus
Projects
Readings
Lectures
Exams
Dates
Grades
Piazza

 

 

 

Note: for each class (after the intro material), 3 or 4 students will be responsible for emailing me (als@cs.umd.edu) with ~4 discussion question on the reading(s) for that day by 6PM the day before the class, and be prepared to ask those questions and help explain the paper to the rest of the class.

Introduction - What and Why? 

1/29 Parallel Computing and Parallel Computers

  • from Lecture Notes

2/3 Applications of Parallel Computing

  • from Lecture Notes

Programming Models

2/5-10 Expressing Parallelism (Explicit Control)

  • V.S. Sunderam, G.A. Geist, J. Dongarra, and R. Manchek, "The PVM Concurrent Computing System: Evolution, Experiences, and Trends", Parallel Computing, 20(4), 1994. [PDF]

  • J. J. Dongarra, S. W. Otto, M. Snir, and D. Walker, "A message passing standard for MPP and workstations," Communications of the ACM, 39(7), 1996, pp. 84-90. [PDF]

2/10-12 Introduction to Debugging Parallel Programs

  • from Taiga Nakamura's lecture notes

2/17 No class - snow

2/19 Expressing Parallelism (Implicit Control) - Victoria Cepeda, Chen Chen, Xi Chen

  • William W. Carlson et al, "Introduction to UPC and Language Specification," CCS-TR-99-157. [PDF]

  • L. Dagum and R. Menon, "OpenMP: An Industry-Standard API for Shared-Memory Programming," IEEE Computational Science & Engineering, 5(1), 1998. [PDF]

2/24 Expressing Parallelism (Hybrids) - Yunteng Di, Mingze Gao, Yu Jin

  • Steve W. Bova et. al., "Parallel Programming with Message Passing and Directives", Computing in Science & Engineering, 3(5), 2001. [PDF]

  • Brent Leback, Michael Wolfe, and Douglas Miles "The PGI Fortran and C99 OpenACC Compilers", Proceedings of Cray User Group (CUG) meeting, 2012. [PDF]

2/26 No class - snow

3/3 Expressing Parallelism (Frameworks) - Ugur Koc, Chester Lam

  • S. Balay, W. D. Gropp, L. C. McInnes, and B. F. Smith, "Efficient Management of Parallelism in Object Oriented Numerical Software Libraries", In E. Arge, A. M. Bruaset, and H. P. Langtangen, editors, Modern Software Tools in Scientific Computing, pages 163--202, Birkhäuser Press, 1997. [PDF]

  • T. Goodale, G. Allen, G. Lanfermann, J. Massó, T. Radke, E. Seidel, and J. Shalf., "The Cactus Framework and Toolkit: Design and Applications", In Proceedings of Vector and Parallel Processing - VECPAR 2002, Springer, 2003. [PDF]

Architectures

3/5 No class - snow

3/10 Shared Memory - Dhiraj Reddy, Mahmoud Sayed, Yoav Segev

  • J. Laudon and D. Lenoski, "The SGI Origin: a ccNUMA highly scalable server," In Proceedings of 1997 International Symposium on Computer Architecture (ISCA '97), May 1997. [PDF]

  • SGI, "Technical Advances in the SGI® UV Architecture™," SGI White paper, 2012. [PDF]

3/12 Message Passing and Communication - Bharat Singh, Octavian-Petru Suciu, Qifei Xu

  • Robert M. Metcalfe and David R. Boggs, "Ethernet: distributed packet switching for local computer networks," Communications of the ACM, (19)7, 1976. [PDF]

  • Mellanox Technologies white paper, "Introduction to InfiniBand.". [PDF]

3/17-19 No class - spring break

3/24 Custom Machines - Yanzhou Liu, Jesse Moll, Mahyar Najibikohnehshahri

  • S.R. Alam, J.A. Kuehn, R.F. Barrett, J.M. Larkin, M.R. Fahey, R. Sankaran, P.H. Worley , "Cray XT4: An Early Evaluation for Petascale Scientific Simulation", In Proceedings of SC'07, Nov. 2007. [PDF]

  • A. Gara, et. al, "Overview of the Blue Gene/L system architecture", IBM Journal of Research and Development, 49(2/3), Fall 2005. [PDF]

3/26 Stream Processing and GPUs - Zhenpeng Zhao, Chen Chen, Xi Chen

  •  A. E. Eichenberger , et. al, "Using advanced compiler technology to exploit the performance of the Cell Broadband Engine architecture", IBM Systems Journal, 45(1), Jan. 2006. [PDF]

  • "Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU", In Proceedings of 2010 International Symposium on Computer Architecture (ISCA), May 2010. [PDF]

3/31 Computational Grids - Yunteng Di, Mingze Gao, Yu Jin

  • I. Foster and C. Kesselman, "Computational Grids", Chapter 2 of The Grid: Blueprint for a New Computing Infrastructure, Morgan Kaufmann, 1999. [PDF]

  • A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, S. Tuecke, "The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets",  Journal of Network and Computer Applications, 23:187-200, 2001. [PDF]

4/2 Clouds - Ugur Koc, Chester Lam, Dhiraj Yoge

  • Jeffrey Dean and Sanjay Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters", In Proceedings of OSDI'04, pp. 137-150  [PDF]

  • Michael Stonebraker, Daniel Abadi, David J. DeWitt, Sam Madden, Erik Paulson, Andrew Pavlo, Alexander Rasin, "MapReduce and Parallel DBMSs: Friends or Foes?", Communications of the ACM, 53(1), Jan. 2010, pp. 64-71. [PDF]

Tools

4/7 Event Ordering and Race Detection - Mahmoud Sayed, Yoav Segev, Bharat Singh

  • L. Lamport, "Time, Clocks, and the Ordering of Events in a Distributed System", Communications of the ACM, 21(7), 1978, pp. 558-564. [PDF]

  • S. Savage, M. Burrows, G. Nelson, P. Sobalvarro, and T. Anderson, "Eraser: A Dynamic Data Race Detector for Multi-Threaded Programs", In Proceedings of the 16th Symposium on Operating Systems Principles, ACM Press, Oct. 1997. [PDF]

4/9 Data Collection and Instrumentation - Octavian-Petru Suciu, Qifei Xu, Yanzhou Liu

  • Nicholas Nethercote and Julian Seward, "Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation",  In Proceedings of  the 2007 ACM/SIGPLAN Conference on Programming Language Design and Implementation (PLDI), June 2007. [PDF]

  • B. R. Buck and J.K. Hollingsworth , "An API for Runtime Code Patching," International Journal of High Performance Computing Applications, 14 (4), Winter 2000, pp. 317-329. [PDF]

4/14 Cache Tools - Jesse Moll, Mahyar Najibikohnehshahri, Zhenpeng Zhao

  • J. Mellor-Crummey, D. Whalley, and K. Kennedy,  "Improving Memory Hierarchy Performance for Irregular Applications Using Data and Computation Reorderings", International Journal of Parallel Programming, 29(3), June 2001. [PDF]

  • Margaret Martonosi, Anoop Gupta, Thomas Anderson, "MemSpy: analyzing memory system bottlenecks in programs", ACM SIGMETRICS Performance Evaluation Review, 20(1), 1992. [PDF]

4/16 Midterm Exam

4/21 Runtime Parallelization - Chen Chen, Xi Chen, Yunteng Di

  • S.J. Fink, S.R. Kohn, and S.B. Baden, "Efficient Run-time Support for Irregular Block-Structured Applications", Journal of Parallel and Distributed Computing, 50(1), 1998. [PDF]

  • G. Agrawal, A. Sussman, and J. Saltz, "An Integrated Runtime and Compile-time Approach for Parallelizing Structured and Block Structured Applications", IEEE Transactions on Parallel and Distributed Systems, 6(7), 1995. [PDF]

Systems Issues

4/23 Finding Idle Cycles - Mingze Gao, Yu Jin, Ugur Koc

  • M. Litzkow, M. Livny, and M. Mutka, "Condor - A Hunter of Idle Workstations", In Proceedings of  International Conference on Distributed Computing Systems, June 1988, pp. 104-111. [PDF]

      For a more up-to-date detailed history of the Condor project, see:

    • D. Thain, T. Tannenbaum, and M. Livny " Distributed Computing in Practice: The Condor Experience", Concurrency and Computation: Practice and Experience , Vol. 17, Nos. 2-4, 2005. [PDF]

  • David P. Anderson, Carl Christensen and Bruce Allen, "Designing a Runtime System for Volunteer Computing", In Proceedings of SC'06, November 2006. [PDF]

4/28 Scheduling - Batch Queues - Chester Lam, Dhiraj Yoge, Mahmoud Sayed

  • D. G. Feitelson and A. M. a. Weil, "Utilization and Predictability in Scheduling the IBM SP2 with Backfilling", 12th International Parallel Processing Symposium, April 1998. Use this extended form [PDF]

  • J. Weinberg and A. Snavely, "Symbiotic Space-Sharing on SDSC's DataStar System",  12th Workshop on Job Scheduling Strategies for Parallel Processing, 2006. [PDF]

4/30 Parallel I/O - Yoav Segev, Bharat Singh, Octavian-Petru Suciu, Mahyar Najibikohnehshahri

  • Terry Jones, Alice Koniges, and R. Kim Yates, "Performance of the IBM General Parallel File System", In Proceedings of 14th International Parallel and Distributed Processing Symposium (IPDPS'00), April 2000. [PDF]

  • A. Acharya, M. Uysal, and J. Saltz, "Active Disks: Programming Model, Algorithms and Evaluation", In Proceedings of Eighth International Conference on Architectural Support for Programming Languages and Operating Systems, October 1998. [PDF]

Applications

5/5 Applications - Qifei Xu, Yanzhou Liu, Jesse Moll, Zhenpeng Zhao

  • U. Catalyurek, M. Beynon, C. Chang, T. Kurc, A. Sussman, and J. Saltz, "The Virtual Microscope",  IEEE Transactions on Information Technology in Biomedicine, Vol. 7, No. 4, 2003. [PDF]

  • David E. Shaw et. al., "Millisecond-scale molecular dynamics simulations on Anton", Proceedings of SC'09, November 2009. [PDF]

5/7 Project Demos

5/12 SC14 Gordon Bell award finalist

  • J. Bedorf, E. Gaburov, M. Fujii, K. Nitadori, T. Ishiyama, S. Zwart,, "24.77 Pflops on a Gravitational Tree-Code to Simulate the Milky Way Galaxy with 18600 GPUs",  Proceedings of SC14, November 2014. [PDF]

 

Web Accessibility