|
A General Overview of Parallel Processing Fall 1998 |
|||
|
|
|||
| Mark Pacifico | mjpac@wam.umd.edu | Mike Merrill | mwm@glue.umd.edu |
| WHAT IS PARALLELISM? | |||
| Parallelism is the concept of solving a problem by dividing the task required into several pieces that can be executed at the same time (in parallel) rather than executing the entire task linearly. Parallel computing (or parallel processing or multiprocessing) involves using the concept of parallelism in the design of a computer architecture. Multiprocessor computers are capable of solving particularly large problems much more quickly than their single processor counterparts because they divide the work necessary to find the solution up and exectue the separate groups of instructions on several different processors at the same time. They then combine their results to yield an answer to the problem. In today's world, most personal and business computers do not take advantage of parallel processing because the uniprocessor machines of the day are adequately fast and efficient. | |||
| WHY IS IT IMPORTANT? | |||
| How important can parallelism be if our computers are already fast enough? Sure, supercomputers might need it to solve extremely complex problems such as determining the climate in 50 years, or breaking a cryptographic code, but most of us don't need it. The answer is that parallelism is very important. Although architectural innovation in uniprocessor technology has been steadily contributing to a rapid rate of processor performance growth since 1985, it is evident that this trend will not continue forever. The fact is, many uniprocessor architectures already take advantage of several of the tricks of multiprocessing that speed up execution time. Because the speed of these uniprocessors is ultimately limited by the speed at which light can travel, the processors themselves are forced to get smaller and smaller. In order to continue at such a high rate of improvement, our computers will eventually have to be microscopic in order to achieve the performance we desire. The alternative is to use parallelism to decrease execution time and therefore increase the speed of our machines. Nobody knows how long it will take, but it is a safe bet that parallelism will eventually find its way into the architecture of nearly every computer that wants to run quickly in the future. | |||
| SIMD/MIMD | |||
| The idea of using parallelism in computer architectures to increase performance and improve availability has been around since the early days of computing. All computers can be placed into one of four categories if you look at the parallelism in the instruction and the data streams called for by the instructions at the most constrained component of the machine. Single instruction stream, single data stream (SISD) computers are the typical uniprocessor machines you and I use all the time. Single instruction, multiple data stream (SIMD) computers are multiprocessor machines that execute the same instruction with multiple processors using different data streams. Each processor has its own data memory, but there is a single instruction memory and control processor that fetches and dispatches instructions. SIMD processors are typically special purpose, since full generality is not required. Multiple instruction, single data stream (MISD) computers are yet to be built commercially, but may be in the future. Multiple instruction, multiple data stream (MIMD) computers often use off the shelf microprocessors. Each processor fetches its own instructions and operates on its own data. Not all machines fall into these categories, as there are several that combine features of more than one type, but in general these are fairly good guidelines. Many of the early model multiprocessors were SIMD, and there are still companies making SIMD computers today. However, the focus has shifted towards MIMD architectures because they make good general purpose microprocessors, and these tend to be more inexpensive and cost efficient than those that are custom made. In fact, almost all MIMDs built today use the same microprocessors found in workstations and small servers. MIMDs also offer flexibility, and can function as single-user machines running a single high-performance application, multiprogrammed machines running several tasks simultaneously, or some combination of both. | |||
| WHAT ABOUT MEMORY? | |||
| In MIMD machines, memory can be arranged in a number of ways. There are two types of MIMDs: centralized shared-memory architectures, and distributed memory architectures. The first type is much more popular than the second type, but for a large number of processors (more than 12 or so) the second type is really the only option. The centralized architecture involves one main memory, and several processors with separate caches that all have access to it. The distibuted architecture involves an interconnection network that provides communication among the separate processors, each with their own cache and slice of main memory. There are other memory arrangements that are variations of these two, but they can get complex and confusing so we will leave them alone for now. | |||
|
|
|||
|
|
|||