Run Time Parallelization And Scheduling Of Loops

This paper studies run-time methods to automatically parallelize and schedule iterations of a do loop in certain cases, where compile-time information is inadequate. The methods presented in this paper involve execution time preprocessing of the loop. At compile-time, these methods set up the framework for performing a loop dependency analysis. At run-time, wavefronts of concurrently executable loop iterations are identified. Using this wavefront information, loop iterations are reordered for increased parallelism. The authors utilize symbolic transformation rules to produce: (1) inspector procedures that perform execution time preprocessing and (2) executors or transformed versions of source code loop structures. These transformed loop structures carry out the calculations planned in the inspector procedures. They present performance results from experiments conducted on the Encore Multimax. These results illustrate that run-time reordering of loop indices can have a significant impact on performance. Furthermore, the overheads associated with this type of reordering are amortized when the loop is executed several times with the same dependency structure. (kr).

I Unidimensional Problems.- 1 Scheduling DAGs without Communications.- 2 Scheduling DAGs with Communications.- 3 Cyclic Scheduling.- II Multidimensional Problems.- 4 Systems of Uniform Recurrence Equations.- 5 Parallelism Detection in Nested Loops.

The second half of the 1970s was marked with impressive advances in array/vector architectures and vectorization techniques and compilers. This progress continued with a particular focus on vector machines until the middle of the 1980s. The major ity of supercomputers during this period were register-to-register (Cray 1) or memory-to-memory (CDC Cyber 205) vector (pipelined) machines. However, the increasing demand for higher computational rates lead naturally to parallel comput ers and software. Through the replication of autonomous processors in a coordinated system, one can skip over performance barriers due technology limitations. In princi ple, parallelism offers unlimited performance potential. Nevertheless, it is very difficult to realize this performance potential in practice. So far, we have seen only the tip of the iceberg called "parallel machines and parallel programming". Parallel programming in particular is a rapidly evolving art and, at present, highly empirical. In this book we discuss several aspects of parallel programming and parallelizing compilers. Instead of trying to develop parallel programming methodologies and paradigms, we often focus on more advanced topics assuming that the reader has an adequate background in parallel processing. The book is organized in three main parts. In the first part (Chapters 1 and 2) we set the stage and focus on program transformations and parallelizing compilers. The second part of this book (Chapters 3 and 4) discusses scheduling for parallel machines from the practical point of view macro and microtasking and supporting environments). Finally, the last part (Le.

Containing over 300 entries in an A-Z format, the Encyclopedia of Parallel Computing provides easy, intuitive access to relevant information for professionals and researchers seeking access to any aspect within the broad field of parallel computing. Topics for this comprehensive reference were selected, written, and peer-reviewed by an international pool of distinguished researchers in the field. The Encyclopedia is broad in scope, covering machine organization, programming languages, algorithms, and applications. Within each area, concepts, designs, and specific implementations are presented. The highly-structured essays in this work comprise synonyms, a definition and discussion of the topic, bibliographies, and links to related literature. Extensive cross-references to other entries within the Encyclopedia support efficient, user-friendly searchers for immediate access to useful information. Key concepts presented in the Encyclopedia of Parallel Computing include; laws and metrics; specific numerical and non-numerical algorithms; asynchronous algorithms; libraries of subroutines; benchmark suites; applications; sequential consistency and cache coherency; machine classes such as clusters, shared-memory multiprocessors, special-purpose machines and dataflow machines; specific machines such as Cray supercomputers, IBM’s cell processor and Intel’s multicore machines; race detection and auto parallelization; parallel programming languages, synchronization primitives, collective operations, message passing libraries, checkpointing, and operating systems. Topics covered: Speedup, Efficiency, Isoefficiency, Redundancy, Amdahls law, Computer Architecture Concepts, Parallel Machine Designs, Benmarks, Parallel Programming concepts & design, Algorithms, Parallel applications. This authoritative reference will be published in two formats: print and online. The online edition features hyperlinks to cross-references and to additional significant research. Related Subjects: supercomputing, high-performance computing, distributed computing

This book introduces new compilation techniques, using the polyhedron model for the resource-adaptive parallel execution of loop programs on massively parallel processor arrays. The authors show how to compute optimal symbolic assignments and parallel schedules of loop iterations at compile time, for cases where the number of available cores becomes known only at runtime. The compile/runtime symbolic parallelization approach the authors describe reduces significantly the runtime overhead, compared to dynamic or just‐in-time compilation. The new, on‐demand fault‐tolerant loop processing approach described in this book protects loop nests for parallel execution against soft errors.

The IEEE Third International Conference on Algorithms and Architectures for Parallel Processing (ICA3PP-97) will be held in Melbourne, Australia from December 8th to 12th, 1997. The purpose of this important conference is to bring together developers and researchers from universities, industry and government to advance science and technology in distributed and parallel systems and processing.

Abstract: "This thesis examines the effect of run-time overhead on the performance of parallel loops on shared-memory multiprocessors. Parallel execution time and speedup are examined as a function of both the loop scheduling strategy and the barrier synchronization mechanism. Analytic models for parallel loop performance are developed and compared to simulation results, and are found to be in good agreement. The models take into account not only deterministic runtime overhead due to scheduling and synchronization, but also random variations in loop iteration execution times with both normal and exponential distributions

We make a detailed examination of the performance achieved by a Krylov space sparse linear system solver that uses incompletely factored matrices for preconditioners. We compared two related mechanisms for parallelizing the computationally critical sparse triangular solves and sparse numeric incomplete factorizations on a range of test problems. From these comparisons we drew several interesting conclusions about methods that can be used to parallelize loops of the type found here. The performance we obtain is brought into perspective by comparison with timing results from a Cray X/MP supercomputer. Performance on an Encore Multimax/320 with relatively modest computational capabilities comes within a small factor of the performance on a comparable code run on a Cray X/MP. (KR).