Multi Armed Bandit Allocation Indices With A Foreword By Peter Whittle

In 1989 the first edition of this book set out Gittins' pioneering index solution to the multi-armed bandit problem and his subsequent investigation of a wide of sequential resource allocation and stochastic scheduling problems. Since then there has been a remarkable flowering of new insights, generalizations and applications, to which Glazebrook and Weber have made major contributions. This second edition brings the story up to date. There are new chapters on the achievable region approach to stochastic optimization problems, the construction of performance bounds for suboptimal policies, Whittle's restless bandits, and the use of Lagrangian relaxation in the construction and evaluation of index policies. Some of the many varied proofs of the index theorem are discussed along with the insights that they provide. Many contemporary applications are surveyed, and over 150 new references are included. Over the past 40 years the Gittins index has helped theoreticians and practitioners to address a huge variety of problems within chemometrics, economics, engineering, numerical analysis, operational research, probability, statistics and website design. This new edition will be an important resource for others wishing to use this approach.

Statisticians are familiar with bandit problems, operations researchers with scheduling programs, and economists with problems of resource allocation. For most of these problems, accurate solutions cannot be obtained unless the problem is small-scale. However, Gittins and Jones showed in 1974 that there is a large class of allocation problems for which the optimal solution is expressible in terms of a priority index that can be calculated. This book is the first definitive account of the theory and applications of this index, which has become known as the Gittens index. Includes 22 previously unpublished tables of index values.

In this second book of what will be a four-volume series, the authors present, in a mathematically rigorous way, the essential foundations of both the theory and practice of algorithms, approximation, and optimization—essential topics in modern applied and computational mathematics. This material is the introductory framework upon which algorithm analysis, optimization, probability, statistics, machine learning, and control theory are built. This text gives a unified treatment of several topics that do not usually appear together: the theory and analysis of algorithms for mathematicians and data science students; probability and its applications; the theory and applications of approximation, including Fourier series, wavelets, and polynomial approximation; and the theory and practice of optimization, including dynamic optimization. When used in concert with the free supplemental lab materials, Foundations of Applied Mathematics, Volume 2: Algorithms, Approximation, Optimization teaches not only the theory but also the computational practice of modern mathematical methods. Exercises and examples build upon each other in a way that continually reinforces previous ideas, allowing students to retain learned concepts while achieving a greater depth. The mathematically rigorous lab content guides students to technical proficiency and answers the age-old question “When am I going to use this?” This textbook is geared toward advanced undergraduate and beginning graduate students in mathematics, data science, and machine learning.

This book has exerted a continuing appeal since its original publication in 1970. It develops the theory of probability from axioms on the expectation functional rather than on probability measure, demonstrates that the standard theory unrolls more naturally and economically this way, and that applications of real interest can be addressed almost immediately. A secondary aim of the original text was to introduce fresh examples and convincing applications, and that aim is continued in this edition, a general revision plus the addition of chapters giving an economical introduction to dynamic programming, that is then applied to the allocation problems represented by portfolio selection and the multi-armed bandit. The investment theme is continued with a critical investigation of the concept of risk-free'trading and the associated Black-Sholes formula, while another new chapter develops the basic ideas of large deviations. The book may be seen as an introduction to probability for students with a basic mathematical facility, covering the standard material, but different in that it is unified by its theme and covers an unusual range of modern applications.

A textbook for an introductory undergraduate course in probability theory, first published in 1970, and revised in 1976. The novelty of the approach is its basis on the subject's expectation rather than on probability measures. Assumes a fair degree of mathematical sophistication. Annotation copyrighted by Book News, Inc., Portland, OR

A comprehensive and rigorous introduction for graduate students and researchers, with applications in sequential decision-making problems.