Jovian-2 [1] is a parallel I/O library developed at the University of Maryland. It presents a simple interface to the user, similar in function to the POSIX lio_listio() routines, where multiple requests for disk I/O can be handled in a single call. Jovian-2 consists of two separate threads when running. The first thread, called the client, is just the main thread that the user application is running in. The server thread, is separate and only used by the Jovian-2 server to process local and remote requests. The actual disk reads and writes are made through standard UNIX asynchronous I/O services, writing to a standard file-system on each of the nodes.
Part of this work on the Alphas was getting away from the proprietary IBM Message Passing Library (MPL) used on the SP-2 machine that Jovian-2 was initially developed on. For reasons of availability and portability, the standard Message Passing Interface (MPI) library was chosen [2]. Only simple non-blocking sends and receives are used for communication between nodes, so the entire complexity of MPI is not really needed. Since MPI is a specification of a standard and not a piece of software, several implementations are available. LAM/MPI version 6.0 was chosen and installed for this work, but in theory any implementation adhering to the MPI standard should work with little or no changes. MPICH version 1.0.12 was also used during development. The LAM/MPI implementation has been shown to be stable and perform well for large messages [5], and is shown to outperform MPICH in many benchmarks [3].
The library can be used many ways, but in keeping with my initial goal of following the spirit of commodity hardware and non-dedicated servers, the library will be configured to act in a peer-to-peer fashion as shown in Figure 1(b) from [1]. This means simply, that each node's process will act as both a client as well as a server. This introduces the problem of arbitration between the server and client portion of each node. The server is a separate thread that fills requests from either the local node or any of the other nodes in the parallel job. This type of configuration is markedly different from other systems where entire nodes are dedicated to being I/O servers. Such system work well, but many cases can be run where that number is either too few or too many I/O nodes. This can be CPU wasteful or a bottleneck, respectively, thus the peer-to-peer style deserves attention.
: Configurations for (a) client-server and (b) peer-to-peer
execution models
