Applications Of Nonstandard Analysis To Markov Processes And Statistical Decision Theory

We use nonstandard analysis to significantly generalize the well-known Markov chain ergodic theorem and establish a fundamentally new complete class theorem, making progress on two core problems in stochastic process theory and statistical decision theory, respectively. In the first part, we study the ergodicity of time-homogenous Markov processes. A time-homogeneous Markov process with stationary distribution $\pi$ is said to be ergodic if its transition probability converges to $\pi$ in total variation distance. In the most general setting of continuous-time Markov processes with general state spaces, there are few results characterizing the ergodicity of the underlying Markov processes. Using the method of nonstandard analysis, for every standard Markov process $\{X_t\}_{t\geq 0}$, we construct a nonstandard Markov process $\{X'_t\}_{t\in T}$ that inherits most of the key properties of $\{X_t\}_{t\geq 0}$ hence establishing the ergodicity without technical conditions, such as on drift or skeleton chains. In the second part, we study the relationship between frequentist and Bayesian optimality, extending the line of work initiated by Wald in the 1940's. Existing results are subject to technical conditions that rule out semi-parametric decision problems and generally rule out non-parametric ones. Using nonstandard analysis, we show that, among decision procedures with finite risk functions, a decision procedure is extended admissible if and only if its extension has infinitesimal excess Bayes risk. The result holds in complete generality, i.e, without regularity conditions or restrictions on the model or the loss function. This nonstandard characterization of extended admissibility also generates a purely standard theorem: when risk functions are continuous on a compact Hausdorff parameter space, a procedure is extended admissible if and only if it is Bayes.

This book is the result of the International Symposium on Semi Markov Processes and their Applications held on June 4-7, 1984 at the Universite Libre de Bruxelles with the help of the FNRS (Fonds National de la Recherche Scientifique, Belgium), the Ministere de l'Education Nationale (Belgium) and the Bernoulli Society for Mathe matical Statistics and Probability. This international meeting was planned to make a state of the art for the area of semi-Markov theory and its applications, to bring together researchers in this field and to create a platform for open and thorough discussion. Main themes of the Symposium are the first ten sections of this book. The last section presented here gives an exhaustive biblio graphy on semi-Markov processes for the last ten years. Papers selected for this book are all invited papers and in addition some contributed papers retained after strong refereeing. Sections are I. Markov additive processes and regenerative systems II. Semi-Markov decision processes III. Algorithmic and computer-oriented approach IV. Semi-Markov models in economy and insurance V. Semi-Markov processes and reliability theory VI. Simulation and statistics for semi-Markov processes VII. Semi-Markov processes and queueing theory VIII. Branching IX. Applications in medicine X. Applications in other fields v PREFACE XI. A second bibliography on semi-Markov processes It is interesting to quote that sections IV to X represent a good sample of the main applications of semi-Markov processes i. e.

Non-Homogeneous Markov Chains and Systems: Theory and Applications fulfills two principal goals. It is devoted to the study of non-homogeneous Markov chains in the first part, and to the evolution of the theory and applications of non-homogeneous Markov systems (populations) in the second. The book is self-contained, requiring a moderate background in basic probability theory and linear algebra, common to most undergraduate programs in mathematics, statistics, and applied probability. There are some advanced parts, which need measure theory and other advanced mathematics, but the readers are alerted to these so they may focus on the basic results. Features A broad and accessible overview of non-homogeneous Markov chains and systems Fills a significant gap in the current literature A good balance of theory and applications, with advanced mathematical details separated from the main results Many illustrative examples of potential applications from a variety of fields Suitable for use as a course text for postgraduate students of applied probability, or for self-study Potential applications included could lead to other quantitative areas The book is primarily aimed at postgraduate students, researchers, and practitioners in applied probability and statistics, and the presentation has been planned and structured in a way to provide flexibility in topic selection so that the text can be adapted to meet the demands of different course outlines. The text could be used to teach a course to students studying applied probability at a postgraduate level or for self-study. It includes many illustrative examples of potential applications, in order to be useful to researchers from a variety of fields.

This book is addressed to mathematicians working in analysis and its applications. The aim is to provide an understandable introduction to the basic theory of nonstan dard analysis in Part I, and then to illuminate some of its most striking applications. Much of the book, in particular Part I, can be used in a graduate course; problems are posed in all chapters. After Part I, each chapter takes up a different field for the application of nonstandard analysis, beginning with a gentle introduction that even non-experts can read with profit. The remainder of each chapter is then addressed to experts, showing how to use nonstandard analysis in the search for solutions of open problems and how to obtain rich new structures that produce deep insight into the field under consideration. The applications discussed here are in functional analysis including operator theory, probability theory including stochastic processes, and economics including game theory and financial mathematics. In all of these areas, the intuitive notion of an infinitely small or infinitely large quantity plays an essential and helpful role in the creative process. For example, Brownian motion is often thought of as a random walk with infinitesimal increments; the spectrum of a selfadjoint operator is viewed as the set of "almost eigenvalues"; an ideal economy consists of an infinite number of agents each having an infinitesimal influence on the economy. Already at the level of calculus, one often views the integral as an infinitely large sum of infinitesimal quantities.

View the abstract.

Since its inception by Perron and Frobenius, the theory of non-negative matrices has developed enormously and is now being used and extended in applied fields of study as diverse as probability theory, numerical analysis, demography, mathematical economics, and dynamic programming, while its development is still proceeding rapidly as a branch of pure mathematics in its own right. While there are books which cover this or that aspect of the theory, it is nevertheless not uncommon for workers in one or another branch of its development to be unaware of what is known in other branches, even though there is often formal overlap. One of the purposes of this book is to relate several aspects of the theory, insofar as this is possible. The author hopes that the book will be useful to mathematicians; but in particular to the workers in applied fields, so the mathematics has been kept as simple as could be managed. The mathematical requisites for reading it are: some knowledge of real-variable theory, and matrix theory; and a little knowledge of complex-variable; the emphasis is on real-variable methods. (There is only one part of the book, the second part of 55.5, which is of rather specialist interest, and requires deeper knowledge.) Appendices provide brief expositions of those areas of mathematics needed which may be less g- erally known to the average reader.

This invaluable book provides approximately eighty examples illustrating the theory of controlled discrete-time Markov processes. Except for applications of the theory to real-life problems like stock exchange, queues, gambling, optimal search etc, the main attention is paid to counter-intuitive, unexpected properties of optimization problems. Such examples illustrate the importance of conditions imposed in the theorems on Markov Decision Processes. Many of the examples are based upon examples published earlier in journal articles or textbooks while several other examples are new. The aim was to collect them together in one reference book which should be considered as a complement to existing monographs on Markov decision processes. The book is self-contained and unified in presentation. The main theoretical statements and constructions are provided, and particular examples can be read independently of others. Examples in Markov Decision Processes is an essential source of reference for mathematicians and all those who apply the optimal control theory to practical purposes. When studying or using mathematical methods, the researcher must understand what can happen if some of the conditions imposed in rigorous theorems are not satisfied. Many examples confirming the importance of such conditions were published in different journal articles which are often difficult to find. This book brings together examples based upon such sources, along with several new ones. In addition, it indicates the areas where Markov decision processes can be used. Active researchers can refer to this book on applicability of mathematical methods and theorems. It is also suitable reading for graduate and research students where they will better understand the theory.

The theory of Markov Processes has become a powerful tool in partial differential equations and potential theory with important applications to physics. Professor Dynkin has made many profound contributions to the subject and in this volume are collected several of his most important expository and survey articles. The content of these articles has not been covered in any monograph as yet. This account is accessible to graduate students in mathematics and operations research and will be welcomed by all those interested in stochastic processes and their applications.

Some of the possible implications among these comparisons remain open questions. The results in this book establish a new field of investigation for both mathematicians and scientific users interested in the variations among multiple probability distributions.