Readings
CMSC 714:  High Performance Computing

Note: for each class (after the intro material), you must send me email (als@cs.umd.edu) with a discussion question on the reading(s) for that day.

Introduction - What and Why? 

9/1 Parallel Computing and Parallel Computers

• from Lecture Notes - A. Sussman

9/6 Applications of Parallel Computing

• from Lecture Notes - A. Sussman

Programming Models

9/8 & 9/13 Expressing Parallelism (Explicit Control) - A. Sussman & C.-W. Tseng

• V.S. Sunderam, G.A. Geist, J. Dongarra, and R. Manchek, "The PVM Concurrent Computing System: Evolution, Experiences, and Trends", Parallel Computing, 20(4), 1994, pp. 531-545. [PDF]

• J. J. Dongarra, S. W. Otto, M. Snir, and D. Walker, "A message passing standard for MPP and workstations," Communications of the ACM, 39(7), 1996, pp. 84-90. [PDF]

9/15 Expressing Parallelism (Implicit Control) - A. Sussman

• William W. Carlson , et al, “Introduction to UPC and Language Specification”, CCS-TR-99-157. [PDF]

• L. Dagum and R. Menon, "OpenMP: An Industry-Standard API for Shared-Memory Programming," IEEE Computational Science & Engineering, 5(1), 1998, pp. 46-55. [PDF]

9/20-22 More Implicit vs. Explicit Control of Parallelism - A. Sussman

• Kathy Yelick et. al., "Titanium: A High Performance Java Dialect", Concurrency: Practice & Experience, 10(11-13), 1998. [PDF]

• Steve W. Bova et. al., "Parallel Programming with Message Passing and Directives", Computing in Science & Engineering, 3(5), 2001, pp. 22-37, [PDF]

9/27 Expressing Parallelism (Data Layout) - J. Hollingsworth

• Z. Bozkus et. al., "Compiling HPF for Distributed Memory MIMD Computers", Journal of Parallel and Distributed Computing, 21(1), 1994, pp. 15-26.  Here is an extended version for a book chapter [PDF]

• Hitoshi Sakagami, Hitoshi Murai, Yoshiki Seo, and Mitsuo Yokokawa, “14.9 TFLOPS Three-dimensional Fluid Simulation for Fusion Science with HPF on the Earth  Simulator”, Proceedings of SC’02, November 2002. [PDF]

9/29 Other Programming Models - N. Rutar

• Jean-Luc Gaudiot, Tom DeBoni, John Feo, Wim Böhm, Walid Najjar, and Patrick Miller, “The Sisal Model of Functional Programming and its Implementation”, Proceedings of the 2nd AIZU International Symposium on Parallel Algorithms / Architecture Synthesis (pAs '97).  [PDF]

Architectures

10/4 - holiday - no class

10/6 Shared Memory - G. Apitz, R. Farrell

• J. Laudon and D. Lenoski, “The SGI Origin: a ccNUMA highly scalable server”, Proceedings of 1997 International Symposium on Computer Architecture (ISCA '97), pp. 241-51, May 1997 [PDF]

• Alan E Charlesworth, “The Sun Fireplane System Interconnect “, Proceedings of SC’01, November 2001. [PDF]

10/11 Message Passing and Communication - M. Black, H. Sivaramakrishnan

• Fabrizio Petrini. Wu-chun Feng,  Adolfy Hoisie, Salvador Coll, Eitan Frachtenberg, “The Quadrics Network: High-Performance Clustering Technology,” IEEE Micro, Jan-Feb 2002, pp. 46-57. [PDF]

• S. L. Scott, "Synchronization and Communication in the T3E Multiprocessor", Proceedings of ASPLOS VII, Cambridge, MA, Oct. 1996 [PDF]

10/13 - holiday - no class

10/18 Vectors and Threading - G. Jackson, J. Kleint

• Gail Alverson, Preston Briggs, Susan , Simon Kahan, Richard Korry, “Tera hardware-software cooperation”, Proceedings of SC’97, November 1997. [PDF]

• P. H. Worley, and T. H. Dunigan, Jr., Early Performance Evaluation of the Cray X1 at Oak Ridge National Laboratory”, Cray User’s Group Conference, May 2003. [PDF]

 10/20 Computational Grids - N. Madnani, A. Sayeed

• I. Foster and C. Kesselman, "Computational Grids", Chapter 2 of The Grid: Blueprint for a New Computing Infrastructure, Morgan Kaufmann, 1999. [PDF]

• A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, S. Tuecke, "The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets",  Journal of Network and Computer Applications, 23:187-200, 2001. [PDF]

Tools

10/25 Event Ordering and Race Detection - K. Berlin, S. Koren

• L. Lamport, "Time, Clocks, and the Ordering of Events in a Distributed System", Communications of the ACM, 21(7), 1978, pp. 558-564. [PDF]

• S. Savage, M. Burrows, G. Nelson, P. Sobalvarro, and T. Anderson, "Eraser: A Dynamic Data Race Detector for Multi-Threaded Programs", In Proceedings of the 16th Symposium on Operating Systems Principles, ACM Press, Oct. 1997. [PDF]

10/27 Performance Metrics - S. Angiuoli, S. Wilson

• A. J. Goldberg and J. L. Hennessy, "Performance Debugging Shared Memory Multiprocessor Programs with MTOOL", In Proceedings of Supercomputing'91, pp. 481-490, Nov. 1991. [PDF]

• J. K. Hollingsworth, "Critical Path Profiling of Message Passing and Shared-memory Programs", IEEE Transactions on Parallel and Distributed Computing, 9(10), 1998, pp. 1029-1040. [PDF]

11/1 Data Collection and Instrumentation - C.-C. Shen, S. Byrd

• J. R. Larus and E. Schnarr, "EEL: Machine-Independent Executable Editing", In Proceedings of the 1995 SIGPLAN Conference on Programming Language Design and Implementation, pages 291-300, June 1995. [Postscript]

• B. R. Buck and J.K. Hollingsworth , “An API for Runtime Code Patching,” International Journal of High Performance Computing Applications, 14 (4), Winter 2000, pp. 317-329. [PDF]

11/3 Scheduling - Short Term - G. Jackson, S. Saha

• Y. Zhang, A. Sivasubramaniam, J. Moreira, and H. Franke, "Impact of Workload and System Parameters on Next Generation Cluster Scheduling Mechanisms", IEEE Transactions on Parallel and Distributed Systems, 12(9), Sept. 2001, pp. 967-985.  [PDF]

• A.C. Dusseau, R.H. Arpaci, D.E. Culler, "Effective Distributed Scheduling of Parallel Workloads", In Proceedings of ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, ACM Press, May 1996. [PDF]

11/8  Performance Tools - J. Kim, H. Shahri

• L.A. De Rose, D.A. Reed, "SvPablo: A Multi-Language Architecture-Independent Performance Analysis System", In Proceedings of the 1999 International Conference on Parallel Processing, Sept. 1999. [PDF]

• B.P. Miller, M.D. Callaghan, J.M. Cargille, J.K. Hollingsworth, R.B. Irvin, K.L. Karavanic, K. Kunchithapadam, and T. Newhall, "The Paradyn Parallel Performance Measurement Tools", IEEE Computer,  28(11), Nov. 1995, pp. 37-46. [PDF]

11/10 Computational Steering - A. Sliva, J. Hsu

• W. Gu, G. Eisenhauer, E. Kraemer, K. Schwan, J. Stasko, J. Vetter, and N. Mallavurupu, "Falcon: On-line Monitoring and Steering of Large-Scale Parallel Programs", In Proceedings of Frontiers '95. Feb. 1995, IEEE Computer Society Press, pp. 422-429. [PDF]

• R. L. Ribler, J. S. Vetter, H. Simitci, and D. A. Reed, "Autopilot: Adaptive Control of Distributed Applications", In Proceedings of International Symposium on High Performance Distributed Computing, July 1998, pp. 172-9. [PDF]

11/15 Cache Tools - S. Alaei, J. Gross

• J. Mellor-Crummey, D. Whalley, and K. Kennedy,  “Improving Memory Hierarchy Performance for Irregular Applications Using Data and Computation Reorderings”, International Journal of Parallel Programming, 29(3), June 2001. [PDF]

• Margaret Martonosi, Anoop Gupta, Thomas Anderson, “MemSpy: analyzing memory system bottlenecks in programs”, ACM SIGMETRICS Performance Evaluation Review, 20(1), 1992. [PDF] [Postscript]

11/17 Runtime Parallelization - A. Sayeed, S. Angiuoli

• S.J. Fink, S.R. Kohn, and S.B. Baden, “Efficient Run-time Support for Irregular Block-Structured Applications”, Journal of Parallel and Distributed Computing, 50(1), 1998. [PDF]

• G. Agrawal, A. Sussman, and J. Saltz, “An Integrated Runtime and Compile-time Approach for Parallelizing Structured and Block Structured Applications”, IEEE Transactions on Parallel and Distributed Computing, 6(7), 1995. [PDF]

Systems Issues

11/22 Grid Support - N. Rutar, S. Koren

• M. Litzkow, M. Livny, and M. Mutka, "Condor - A Hunter of Idle Workstations", In Proceedings of  International Conference on Distributed Computing Systems, June 1988, pp. 104-111. [PDF]

• David Anderson, "BOINC: A System for Public-Resource Computing and Storage", In Proceedings of the Fifth International Workshop on Grid Computing - GRID 2004, Nov. 2004. [PDF]

11/29 - exam

12/1 Parallel I/O - M. Black, R. Farrell

• Terry Jones, Alice Koniges, and R. Kim Yates, “Performance of the IBM General Parallel File System”, In Proceedings of 14th International Parallel and Distributed Processing Symposium (IPDPS'00), April 2000. [PDF]

• A. Acharya, M. Uysal, and J. Saltz, "Active Disks: Programming Model, Algorithms and Evaluation", In Proceedings of Eighth International Conference on Architectural Support for Programming Languages and Operating Systems. Oct.1998. [PDF]

12/6 & 12/8 - project demos

12/13 Applications - H. Sivaramkrishnan, N. Madnani

• A. Fox, S.D. Gribble, Y. Chawathe, E.A. Brewer, P. Gauthier, “Cluster-based scalable network services”, In Proceedings of ACM Symposium on Operating System Principles (SOSP’97), Oct. 1997, pp. 78-91. [PDF]

• U. Catalyurek, M. Beynon, C. Chang, T. Kurc, A. Sussman, and J. Saltz, "The Virtual Microscope",  IEEE Transactions on Information Technology in Biomedicine, Vol. 7, No. 4, Dec. 2003. [PDF]