Task Scheduling In Parallel Processor Systems

A new model for task scheduling that dramatically improves the efficiency of parallel systems Task scheduling for parallel systems can become a quagmire of heuristics, models, and methods that have been developed over the past decades. The author of this innovative text cuts through the confusion and complexity by presenting a consistent and comprehensive theoretical framework along with realistic parallel system models. These new models, based on an investigation of the concepts and principles underlying task scheduling, take into account heterogeneity, contention for communication resources, and the involvement of the processor in communications. For readers who may be new to task scheduling, the first chapters are essential. They serve as an excellent introduction to programming parallel systems, and they place task scheduling within the context of the program parallelization process. The author then reviews the basics of graph theory, discussing the major graph models used to represent parallel programs. Next, the author introduces his task scheduling framework. He carefully explains the theoretical background of this framework and provides several examples to enable readers to fully understand how it greatly simplifies and, at the same time, enhances the ability to schedule. The second half of the text examines both basic and advanced scheduling techniques, offering readers a thorough understanding of the principles underlying scheduling algorithms. The final two chapters address communication contention in scheduling and processor involvement in communications. Each chapter features exercises that help readers put their new skills into practice. An extensive bibliography leads to additional information for further research. Finally, the use of figures and examples helps readers better visualize and understand complex concepts and processes. Researchers and students in distributed and parallel computer systems will find that this text dramatically improves their ability to schedule tasks accurately and efficiently.

Overview and Goals This book is dedicated to scheduling for parallel processing. Presenting a research ?eld as broad as this one poses considerable dif?culties. Scheduling for parallel computing is an interdisciplinary subject joining many ?elds of science and te- nology. Thus, to understand the scheduling problems and the methods of solving them it is necessary to know the limitations in related areas. Another dif?culty is that the subject of scheduling parallel computations is immense. Even simple search in bibliographical databases reveals thousands of publications on this topic. The - versity in understanding scheduling problems is so great that it seems impossible to juxtapose them in one scheduling taxonomy. Therefore, most of the papers on scheduling for parallel processing refer to one scheduling problem resulting from one way of perceiving the reality. Only a few publications attempt to arrange this ?eld of knowledge systematically. In this book we will follow two guidelines. One guideline is a distinction - tween scheduling models which comprise a set of scheduling problems solved by dedicated algorithms. Thus, the aim of this book is to present scheduling models for parallel processing, problems de?ned on the grounds of certain scheduling models, and algorithms solving the scheduling problems. Most of the scheduling problems are combinatorial in nature. Therefore, the second guideline is the methodology of computational complexity theory. Inthisbookwepresentfourexamplesofschedulingmodels. Wewillgodeepinto the models, problems, and algorithms so that after acquiring some understanding of them we will attempt to draw conclusions on their mutual relationships.

In this study, we consider a non-preemtive task scheduling problem on iden- tical and unrelated parallel processor systems. We are interested in the objective function that minimizes the maximum of the completion time of the entire set of tasks (i.e makespan) so as to ensure a good load balance on the parallel proces- sors. We consider three different task characteristics to the classical task scheduling problem that has a set of n independent tasks to be assigned to m parallel processors.

Multiple processor systems are an important class of parallel systems. Over the years, several architectures have been proposed to build such systems to satisfy the requirements of high performance computing. These architectures span a wide variety of system types. At the low end of the spectrum, we can build a small, shared-memory parallel system with tens of processors. These systems typically use a bus to interconnect the processors and memory. Such systems, for example, are becoming commonplace in high-performance graph ics workstations. These systems are called uniform memory access (UMA) multiprocessors because they provide uniform access of memory to all pro cessors. These systems provide a single address space, which is preferred by programmers. This architecture, however, cannot be extended even to medium systems with hundreds of processors due to bus bandwidth limitations. To scale systems to medium range i. e. , to hundreds of processors, non-bus interconnection networks have been proposed. These systems, for example, use a multistage dynamic interconnection network. Such systems also provide global, shared memory like the UMA systems. However, they introduce local and remote memories, which lead to non-uniform memory access (NUMA) architecture. Distributed-memory architecture is used for systems with thousands of pro cessors. These systems differ from the shared-memory architectures in that there is no globally accessible shared memory. Instead, they use message pass ing to facilitate communication among the processors. As a result, they do not provide single address space.

Multiple processor systems are an important class of parallel systems. Over the years, several architectures have been proposed to build such systems to satisfy the requirements of high performance computing. These architectures span a wide variety of system types. At the low end of the spectrum, we can build a small, shared-memory parallel system with tens of processors. These systems typically use a bus to interconnect the processors and memory. Such systems, for example, are becoming commonplace in high-performance graph ics workstations. These systems are called uniform memory access (UMA) multiprocessors because they provide uniform access of memory to all pro cessors. These systems provide a single address space, which is preferred by programmers. This architecture, however, cannot be extended even to medium systems with hundreds of processors due to bus bandwidth limitations. To scale systems to medium range i. e. , to hundreds of processors, non-bus interconnection networks have been proposed. These systems, for example, use a multistage dynamic interconnection network. Such systems also provide global, shared memory like the UMA systems. However, they introduce local and remote memories, which lead to non-uniform memory access (NUMA) architecture. Distributed-memory architecture is used for systems with thousands of pro cessors. These systems differ from the shared-memory architectures in that there is no globally accessible shared memory. Instead, they use message pass ing to facilitate communication among the processors. As a result, they do not provide single address space.

This book constitutes the strictly refereed post-workshop proceedings of the International Workshop on Job Scheduling Strategies for Parallel Processing, held in conjunction with IPPS '96 symposium in Honolulu, Hawaii, in April 1996. The book presents 15 thoroughly revised full papers accepted for inclusion on the basis of the reports of at least five program committee members. The volume is a highly competent contribution to advancing the state-of-the-art in the area of job scheduling for parallel supercomputers. Among the topics addressed are job scheduler, workload evolution, gang scheduling, multiprocessor scheduling, parallel processor allocation, and distributed memory environments.

This book presents task-scheduling techniques for emerging complex parallel architectures including heterogeneous multi-core architectures, warehouse-scale datacenters, and distributed big data processing systems. The demand for high computational capacity has led to the growing popularity of multicore processors, which have become the mainstream in both the research and real-world settings. Yet to date, there is no book exploring the current task-scheduling techniques for the emerging complex parallel architectures. Addressing this gap, the book discusses state-of-the-art task-scheduling techniques that are optimized for different architectures, and which can be directly applied in real parallel systems. Further, the book provides an overview of the latest advances in task-scheduling policies in parallel architectures, and will help readers understand and overcome current and emerging issues in this field.

This book constitutes the thoroughly refereed postproceedings of the 10th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2004, held in New York, NY in June 2004. The 15 revised full research papers presented together with a report on scheduling on the Top 50 machines went through two rounds of reviewing and improvement. Various current issues in job scheduling and load balancing are addressed in the context of computing clusters, parallel and distributed systems, multi-processor systems, and supercomputers.

Parallel processing has seen incredible growth with increasingly complex scientific & engg applications coming to their realm.Task scheduling, a key issue for its success,& design of efficient algorithms play a crucial role in it.Due to NP-completeness of the problem research efforts are mainly focussed at heuristic-based approaches to generate near-optimal schedules within reasonable time & resource constraints.Most of the heuristics, however, ignore practical aspect and/or compromise too much on complexity or performance.This monograph provides an in-depth insight into these issues.An overview of state-of-the-art scheduling, reflecting changing paradigms, is also provided.In addition, for scheduling DAG structured applications,3 efficient algorithms are presented & analyzed for homogeneous, heterogeneous & mixed-parallel computing environments.Selective-duplication heuristic is introduced and its usefulness in dealing with IPC overhead is shown in comparison to list, clustering and duplication-based heuristics.An A-cube performance model is suggested to comprehensively study behavior of algorithm, application & architecture in heterogeneous environment.

This book constitutes the thoroughly refereed post-conference proceedings of the 18th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2014, held in Phoenix, AZ, USA, in May 2014. The 9 revised full papers presented were carefully reviewed and selected from 24 submissions. The papers cover the following topics: single-core parallelism; moving to distributed-memory, larger-scale systems, scheduling fairness; and parallel job scheduling.