Numerical Analysis And Parallel Processing

This highly acclaimed work, first published by Prentice Hall in 1989, is a comprehensive and theoretically sound treatment of parallel and distributed numerical methods. It focuses on algorithms that are naturally suited for massive parallelization, and it explores the fundamental convergence, rate of convergence, communication, and synchronization issues associated with such algorithms. This is an extensive book, which aside from its focus on parallel and distributed algorithms, contains a wealth of material on a broad variety of computation and optimization topics. It is an excellent supplement to several of our other books, including Convex Optimization Algorithms (Athena Scientific, 2015), Nonlinear Programming (Athena Scientific, 1999), Dynamic Programming and Optimal Control (Athena Scientific, 2012), Neuro-Dynamic Programming (Athena Scientific, 1996), and Network Optimization (Athena Scientific, 1998). The on-line edition of the book contains a 95-page solutions manual.

Each week of this three week meeting was a self-contained event, although each had the same underlying theme - the effect of parallel processing on numerical analysis. Each week provided the opportunity for intensive study to broaden participants' research interests or deepen their understanding of topics of which they already had some knowledge. There was also the opportunity for continuing individual research in the stimulating environment created by the presence of several experts of international stature. This volume contains lecture notes for most of the major courses of lectures presented at the meeting; they cover topics in parallel algorithms for large sparse linear systems and optimization, an introductory survey of level-index arithmetic and superconvergence in the finite element method.

This book introduces the basic concepts of parallel and vector computing in the context of an introduction to numerical methods. It contains chapters on parallel and vector matrix multiplication and solution of linear systems by direct and iterative methods. It is suitable for advanced undergraduate and beginning graduate courses in computer science, applied mathematics, and engineering. Ideally, students will have access to a parallel or Vector computer, but the material can be studied profitably in any case. Gives a modern overview of scientific computing including parallel an vector computation Introduces numerical methods for both ordinary and partial differential equations Has considerable discussion of both direct and iterative methods for linear systems of equations, including parallel and vector algorithms Covers most of the main topics for a first course in numerical methods and can serve as a text for this course

This IMA Volume in Mathematics and its Applications ALGORITHMS FOR PARALLEL PROCESSING is based on the proceedings of a workshop that was an integral part of the 1996-97 IMA program on "MATHEMATICS IN HIGH-PERFORMANCE COMPUTING. " The workshop brought together algorithm developers from theory, combinatorics, and scientific computing. The topics ranged over models, linear algebra, sorting, randomization, and graph algorithms and their analysis. We thank Michael T. Heath of University of lllinois at Urbana (Com puter Science), Abhiram Ranade of the Indian Institute of Technology (Computer Science and Engineering), and Robert S. Schreiber of Hewlett Packard Laboratories for their excellent work in organizing the workshop and editing the proceedings. We also take this opportunity to thank the National Science Founda tion (NSF) and the Army Research Office (ARO), whose financial support made the workshop possible. A vner Friedman Robert Gulliver v PREFACE The Workshop on Algorithms for Parallel Processing was held at the IMA September 16 - 20, 1996; it was the first workshop of the IMA year dedicated to the mathematics of high performance computing. The work shop organizers were Abhiram Ranade of The Indian Institute of Tech nology, Bombay, Michael Heath of the University of Illinois, and Robert Schreiber of Hewlett Packard Laboratories. Our idea was to bring together researchers who do innovative, exciting, parallel algorithms research on a wide range of topics, and by sharing insights, problems, tools, and methods to learn something of value from one another.

The use of parallel programming and architectures is essential for simulating and solving problems in modern computational practice. There has been rapid progress in microprocessor architecture, interconnection technology and software devel- ment, which are in?uencing directly the rapid growth of parallel and distributed computing. However, in order to make these bene?ts usable in practice, this dev- opment must be accompanied by progress in the design, analysis and application aspects of parallel algorithms. In particular, new approaches from parallel num- ics are important for solving complex computational problems on parallel and/or distributed systems. The contributions to this book are focused on topics most concerned in the trends of today’s parallel computing. These range from parallel algorithmics, progr- ming, tools, network computing to future parallel computing. Particular attention is paid to parallel numerics: linear algebra, differential equations, numerical integ- tion, number theory and their applications in computer simulations, which together form the kernel of the monograph. We expect that the book will be of interest to scientists working on parallel computing, doctoral students, teachers, engineers and mathematicians dealing with numerical applications and computer simulations of natural phenomena.

This book provides basic and practical techniques of parallel computing and related methods of numerical analysis for researchers who conduct numerical calculation and simulation. Although the techniques provided in this book are field-independent, these methods can be used in fields such as physics, chemistry, biology, earth sciences, space science, meteorology, disaster prevention, and manufacturing. In particular, those who develop software code in these areas will find this book useful. The contents are suitable for graduate students and researchers in computational science rather than novices at programming or informed experts in computer science. Starting with an introduction to the recent trends in computer architecture and parallel processing, Chapter 1 explains the basic knowledge of speedup programs with simple examples of numerical computing. Chapters 2 – 4 detail the basics of parallel programming, the message passing interface (MPI), and OpenMP and discuss hybrid parallelization techniques. Showing an actual example of adaptation, Chapter 5 gives an overview of performance tuning and communication optimizations. To deal with dense matrix calculations, Chapter 6 details the basics and practice of linear algebra calculation libraries BLAS and LAPACK, including some examples that can be easily reproduced by readers using free software. Focusing on sparse matrix calculations, Chapter 7 explains high performance algorithms for numerical linear algebra. Chapter 8 introduces the fast Fourier transform in large-scale systems from the basics. Chapter 9 explains optimization and related topics such as debug methods and version control systems. Chapter 10 discusses techniques for increasing computation accuracy as an essential topic in numerical calculation. This is the first of the two volumes that grew out of a series of lectures in the K computer project in Japan. The second volume will focus on advanced techniques and examples of applications in materials science.