Off Line Approximate Dynamic Programming For The Vehicle Routing Problem With Stochastic Customers And Demands Via Decentralized Decision Making

This book provides a straightforward overview for every researcher interested in stochastic dynamic vehicle routing problems (SDVRPs). The book is written for both the applied researcher looking for suitable solution approaches for particular problems as well as for the theoretical researcher looking for effective and efficient methods of stochastic dynamic optimization and approximate dynamic programming (ADP). To this end, the book contains two parts. In the first part, the general methodology required for modeling and approaching SDVRPs is presented. It presents adapted and new, general anticipatory methods of ADP tailored to the needs of dynamic vehicle routing. Since stochastic dynamic optimization is often complex and may not always be intuitive on first glance, the author accompanies the ADP-methodology with illustrative examples from the field of SDVRPs. The second part of this book then depicts the application of the theory to a specific SDVRP. The process starts from the real-world application. The author describes a SDVRP with stochastic customer requests often addressed in the literature, and then shows in detail how this problem can be modeled as a Markov decision process and presents several anticipatory solution approaches based on ADP. In an extensive computational study, he shows the advantages of the presented approaches compared to conventional heuristics. To allow deep insights in the functionality of ADP, he presents a comprehensive analysis of the ADP approaches.

We present solution methodologies for vehicle routing problems (VRPs) with stochastic demand, with a specific focus on the vehicle routing problem with stochastic demand (VRPSD) and the vehicle routing problem with stochastic demand and duration limits (VRPSDL). The VRPSD and the VRPSDL are fundamental problems underlying many operational challenges in the fields of logistics and supply chain management. We model the VRPSD and the VRPSDL as large-scale Markov decision processes. We develop cyclic-order neighborhoods, a general methodology for solving a broad class of VRPs, and use this technique to obtain static, fixed route policies for the VRPSD. We develop pre-decision, post-decision, and hybrid rollout policies for approximate dynamic programming (ADP). These policies lay a methodological foundation for solving large-scale sequential decision problems and provide a framework for developing dynamic routing policies. Our dynamic rollout policies for the VRPSDL significantly improve upon a method frequently implemented in practice. We also identify circumstances in which our rollout policies appear to offer little or no benefit compared to this benchmark. These observations can guide managerial decision making regarding when the use of our procedures is justifiable. We also demonstrate that our methodology lends itself to real-time implementation, thereby providing a mechanism to make high-quality, dynamic routing decisions for large-scale operations. Finally, we consider a more traditional ADP approach to the VRPSDL by developing a parameterized linear function to approximate the value functions corresponding to our problem formulation. We estimate parameters via a simulation-based algorithm and show that initializing parameter values via our rollout policies leads to significant improvements. However, we conclude that additional research is required to develop a parametric ADP methodology comparable or superior to our rollout policies.

Vehicle routing problems, among the most studied in combinatorial optimization, arise in many practical contexts (freight distribution and collection, transportation, garbage collection, newspaper delivery, etc.). Operations researchers have made significant developments in the algorithms for their solution, and Vehicle Routing: Problems, Methods, and Applications, Second Edition reflects these advances. The text of the new edition is either completely new or significantly revised and provides extensive and complete state-of-the-art coverage of vehicle routing by those who have done most of the innovative research in the area; it emphasizes methodology related to specific classes of vehicle routing problems and, since vehicle routing is used as a benchmark for all new solution techniques, contains a complete overview of current solutions to combinatorial optimization problems. It also includes several chapters on important and emerging applications, such as disaster relief and green vehicle routing.

This paper addresses the capacitated vehicle routing problem with unsplittable stochastic demand (UCVRP), in the limited information regime. We integrate ideas from region partitioning and online bin packing to propose a provably near-optimal and scalable algorithm. Our algorithm performs double partitioning: it first assigns routes to each vehicle a priori and then solves the recourse problem via a highly-efficient online bin-packing algorithm. One salient feature of our algorithm is that each driver only needs to know the information of a small number of customers, even when the total number of customers is extremely large. We characterize the performance of the algorithm and its convergence rate to the optimal offline solution with respect to the amount of customer information through asymptotic analysis. Moreover, we analyze the impact of adding additional overlapping routes and show in heavy traffic scenarios, additional overlapping does not improve the rate of convergence to the optimal solution except by constants. Finally, the effectiveness of the algorithm is further verified through numerical simulations.

A complete and accessible introduction to the real-world applications of approximate dynamic programming With the growing levels of sophistication in modern-day operations, it is vital for practitioners to understand how to approach, model, and solve complex industrial problems. Approximate Dynamic Programming is a result of the author's decades of experience working in large industrial settings to develop practical and high-quality solutions to problems that involve making decisions in the presence of uncertainty. This groundbreaking book uniquely integrates four distinct disciplines—Markov design processes, mathematical programming, simulation, and statistics—to demonstrate how to successfully model and solve a wide range of real-life problems using the techniques of approximate dynamic programming (ADP). The reader is introduced to the three curses of dimensionality that impact complex problems and is also shown how the post-decision state variable allows for the use of classical algorithmic strategies from operations research to treat complex stochastic optimization problems. Designed as an introduction and assuming no prior training in dynamic programming of any form, Approximate Dynamic Programming contains dozens of algorithms that are intended to serve as a starting point in the design of practical solutions for real problems. The book provides detailed coverage of implementation challenges including: modeling complex sequential decision processes under uncertainty, identifying robust policies, designing and estimating value function approximations, choosing effective stepsize rules, and resolving convergence issues. With a focus on modeling and algorithms in conjunction with the language of mainstream operations research, artificial intelligence, and control theory, Approximate Dynamic Programming: Models complex, high-dimensional problems in a natural and practical way, which draws on years of industrial projects Introduces and emphasizes the power of estimating a value function around the post-decision state, allowing solution algorithms to be broken down into three fundamental steps: classical simulation, classical optimization, and classical statistics Presents a thorough discussion of recursive estimation, including fundamental theory and a number of issues that arise in the development of practical algorithms Offers a variety of methods for approximating dynamic programs that have appeared in previous literature, but that have never been presented in the coherent format of a book Motivated by examples from modern-day operations research, Approximate Dynamic Programming is an accessible introduction to dynamic modeling and is also a valuable guide for the development of high-quality solutions to problems that exist in operations research and engineering. The clear and precise presentation of the material makes this an appropriate text for advanced undergraduate and beginning graduate courses, while also serving as a reference for researchers and practitioners. A companion Web site is available for readers, which includes additional exercises, solutions to exercises, and data sets to reinforce the book's main concepts.

In a unified and carefully developed presentation, this book systematically examines recent developments in VRP. The book focuses on a portfolio of significant technical advances that have evolved over the past few years for modeling and solving vehicle routing problems and VRP variations. Reflecting the most recent scholarship, this book is written by one of the top research scholars in Vehicle Routing and is one of the most important books in VRP to be published in recent times.