Compile Time Partitioning And Scheduling Of Parallel Programs Extended Summary

One of the biggest challenges facing language designers and implementors is to develop languages that will be suitable for use on multiprocessors. A wide variety of multiprocessor architectures are currently being built and that variety is expected to increase. Very little software is available to exploit parallelism and speed-up the execution of individual programs. What little work has been done, has focused largely on special-purpose parallelism (e.g. vectorization) or very limited classes of architectures. To understand the issues involved in taking advantage of parallelism and to evaluate architectures, we need to find compilation techniques for fairly general-purpose languages; these techniques need to be adapted to a wide variety of architectures. Only then will it be possible to compare various languages, alternative architectures, and their interaction with different applications. To compile and execute a program in a parallel fashion on a multiprocessor, there are three fundamental problems to be solved: 1) Identification of potential parallelism - discovering parallel computations in the program. 2) Partitioning the program into tasks - each task represents a serially-executed component that can execute in parallel with other tasks. 3) Scheduling the execution of tasks - the tasks must be scheduled for execution on multiple processors.

This book is one of the first to address the problem of forming useful parallelism from potential parallelism and to provide a general solution. The book presents two approaches to automatic partitioning and scheduling so that the same parallel program can be made to execute efficiently on widely different multiprocessors. The first approach is based on a macro dataflow model in which the program is partitioned into tasks at compile time and the tasks are scheduled on processors at run time. The second approach is based on a compile time scheduling model, where both the partitioning and scheduling are performed at compile time. Both approaches have been implemented in partition programs written in the single assignment language SISAL. The inputs to the partitioning and scheduling algorithms are a graphical representation of the parallel program and a list of parameters describing the target multiprocessor. Execution profile information is used to derive compile-time estimates of execution times and data sizes in the program. Both the macro dataflow and compile-time scheduling problems are expressed as optimization problems and are shown to be NP complete in the strong sense. Efficient approximation algorithms for these problems are presented. Finally, the effectiveness of the partitioning and scheduling algorithms is studied by multiprocessor simulations of various SISAL benchmark programs for different target multiprocessor parameters. Vivek Sarkar is a Member of Research Staff at the IBM T. J. Watson Research Center. Partitioning and Scheduling Parallel Programs for Multiprocessing is included in the series Research Monographs in Parallel and Distributed Computing. Copublished with Pitman Publishing.

Partitioning and scheduling techniques are necessary to implement parallel languages on multiprocessors. Multiprocessor performance is maximized when parallelism between tasks is optimally traded off with communication and synchronization overhead. The authors present compile-time partitioning and scheduling techniques to achieve this tradeoff. One of the biggest challenges facing compiler writers is to efficiently implement programming languages on multiprocessors. We need to find compilation techniques for general-purpose parallel languages; these techniques should be adaptable to a wide range of multiprocessor architectures. There are three fundamental problems to be solved when compiling a program for parallel execution on a multiprocessor: 1) Identifying potential parallelism; 2) Partitioning the program into sequential tasks; and 3) Scheduling the concurrent execution of these tasks. This document addresses the latter two problems and suggest that these be solved at compile- time instead of run-time for applications with fairly predictable execution times.

Euro-Par – the European Conference on Parallel Computing – is an international conference series dedicated to the promotion and advancement of all aspects of parallel computing. The major themes can be divided into the broad categories of hardware, software, algorithms, and applications for parallel computing. The objective of Euro-Par is to provide a forum within which to promote the dev- opment of parallel computing both as an industrial technique and an academic discipline, extending the frontiers of both the state of the art and the state of the practice. This is particularlyimportant at a time when parallel computing is undergoing strong and sustained development and experiencing real ind- trial take up. The main audience for and participants in Euro-Par are seen as researchers in academic departments, government laboratories, and industrial organisations. Euro-Par aims to become the primarychoice of such professionals for the presentation of new results in their speci?c areas. Euro-Par is also int- ested in applications that demonstrate the e?ectiveness of the main Euro-Par themes. Euro-Par has its own Internet domain with a permanent web site where the historyof the conference series is described: http://www. euro-par. org. The Euro-Par conference series is sponsored bythe Association of Computer Machineryand the International Federation of Information Processing. Euro-Par 2001 Euro-Par 2001 was organised bythe Universityof Manchester and UMIST.

This book focuses on the future directions of the static scheduling and dynamic load balancing methods in parallel and distributed systems. It provides an overview and a detailed discussion of a wide range of topics from theoretical background to practical, state-of-the-art scheduling and load balancing techniques.