Numerical Methods In Stochastic Control

Stochastic control is a very active area of research. This monograph, written by two leading authorities in the field, has been updated to reflect the latest developments. It covers effective numerical methods for stochastic control problems in continuous time on two levels, that of practice and that of mathematical development. It is broadly accessible for graduate students and researchers.

This book is concerned with numerical methods for stochastic control and optimal stochastic control problems. The random process models of the controlled or uncontrolled stochastic systems are either diffusions or jump diffusions. Stochastic control is a very active area of research and new prob lem formulations and sometimes surprising applications appear regularly. We have chosen forms of the models which cover the great bulk of the for mulations of the continuous time stochastic control problems which have appeared to date. The standard formats are covered, but much emphasis is given to the newer and less well known formulations. The controlled process might be either stopped or absorbed on leaving a constraint set or upon first hitting a target set, or it might be reflected or "projected" from the boundary of a constraining set. In some of the more recent applications of the reflecting boundary problem, for example the so-called heavy traffic approximation problems, the directions of reflection are actually discontin uous. In general, the control might be representable as a bounded function or it might be of the so-called impulsive or singular control types. Both the "drift" and the "variance" might be controlled. The cost functions might be any of the standard types: Discounted, stopped on first exit from a set, finite time, optimal stopping, average cost per unit time over the infinite time interval, and so forth.

The required background is surveyed, and there is an extensive development of methods of approximation and computational algorithms. The book is written on two levels: algorithms and applications, and mathematical proofs. Thus, the ideas should be very accessible to a broad audience."--BOOK JACKET.

The Markov chain approximation methods are widely used for the numerical solution of nonlinear stochastic control problems in continuous time. This book extends the methods to stochastic systems with delays. The book is the first on the subject and will be of great interest to all those who work with stochastic delay equations and whose main interest is either in the use of the algorithms or in the mathematics. An excellent resource for graduate students, researchers, and practitioners, the work may be used as a graduate-level textbook for a special topics course or seminar on numerical methods in stochastic control.

The numerical analysis of stochastic differential equations (SDEs) differs significantly from that of ordinary differential equations. This book provides an easily accessible introduction to SDEs, their applications and the numerical methods to solve such equations. From the reviews: "The authors draw upon their own research and experiences in obviously many disciplines... considerable time has obviously been spent writing this in the simplest language possible." --ZAMP

Stochastic control is a very active area of research and this monograph, written by two leading authorities in the field, has been updated to reflect the latest developments. It covers effective numerical methods for stochastic control problems in continuous time on two levels: that of practice (algorithms and applications) and that of mathematical development. It is broadly accessible for graduate students and researchers.

This book may be regarded as consisting of two parts. In Chapters I-IV we pre sent what we regard as essential topics in an introduction to deterministic optimal control theory. This material has been used by the authors for one semester graduate-level courses at Brown University and the University of Kentucky. The simplest problem in calculus of variations is taken as the point of departure, in Chapter I. Chapters II, III, and IV deal with necessary conditions for an opti mum, existence and regularity theorems for optimal controls, and the method of dynamic programming. The beginning reader may find it useful first to learn the main results, corollaries, and examples. These tend to be found in the earlier parts of each chapter. We have deliberately postponed some difficult technical proofs to later parts of these chapters. In the second part of the book we give an introduction to stochastic optimal control for Markov diffusion processes. Our treatment follows the dynamic pro gramming method, and depends on the intimate relationship between second order partial differential equations of parabolic type and stochastic differential equations. This relationship is reviewed in Chapter V, which may be read inde pendently of Chapters I-IV. Chapter VI is based to a considerable extent on the authors' work in stochastic control since 1961. It also includes two other topics important for applications, namely, the solution to the stochastic linear regulator and the separation principle.

The@ first graduate-level textbook to focus on fundamental aspects of numerical methods for stochastic computations, this book describes the class of numerical methods based on generalized polynomial chaos (gPC). These fast, efficient, and accurate methods are an extension of the classical spectral methods of high-dimensional random spaces. Designed to simulate complex systems subject to random inputs, these methods are widely used in many areas of computer science and engineering. The book introduces polynomial approximation theory and probability theory; describes the basic theory of gPC methods through numerical examples and rigorous development; details the procedure for converting stochastic equations into deterministic ones; using both the Galerkin and collocation approaches; and discusses the distinct differences and challenges arising from high-dimensional problems. The last section is devoted to the application of gPC methods to critical areas such as inverse problems and data assimilation. Ideal for use by graduate students and researchers both in the classroom and for self-study, Numerical Methods for Stochastic Computations provides the required tools for in-depth research related to stochastic computations. The first graduate-level textbook to focus on the fundamentals of numerical methods for stochastic computations Ideal introduction for graduate courses or self-study Fast, efficient, and accurate numerical methods Polynomial approximation theory and probability theory included Basic gPC methods illustrated through examples

Inverse problems of identifying parameters and initial/boundary conditions in deterministic and stochastic partial differential equations constitute a vibrant and emerging research area that has found numerous applications. A related problem of paramount importance is the optimal control problem for stochastic differential equations. This edited volume comprises invited contributions from world-renowned researchers in the subject of control and inverse problems. There are several contributions on optimal control and inverse problems covering different aspects of the theory, numerical methods, and applications. Besides a unified presentation of the most recent and relevant developments, this volume also presents some survey articles to make the material self-contained. To maintain the highest level of scientific quality, all manuscripts have been thoroughly reviewed.