Iterative Optimization In Inverse Problems

Iterative Optimization in Inverse Problems brings together a number of important iterative algorithms for medical imaging, optimization, and statistical estimation. It incorporates recent work that has not appeared in other books and draws on the author’s considerable research in the field, including his recently developed class of SUMMA algorithms. Related to sequential unconstrained minimization methods, the SUMMA class includes a wide range of iterative algorithms well known to researchers in various areas, such as statistics and image processing. Organizing the topics from general to more specific, the book first gives an overview of sequential optimization, the subclasses of auxiliary-function methods, and the SUMMA algorithms. The next three chapters present particular examples in more detail, including barrier- and penalty-function methods, proximal minimization, and forward-backward splitting. The author also focuses on fixed-point algorithms for operators on Euclidean space and then extends the discussion to include distance measures other than the usual Euclidean distance. In the final chapters, specific problems illustrate the use of iterative methods previously discussed. Most chapters contain exercises that introduce new ideas and make the book suitable for self-study. Unifying a variety of seemingly disparate algorithms, the book shows how to derive new properties of algorithms by comparing known properties of other algorithms. This unifying approach also helps researchers—from statisticians working on parameter estimation to image scientists processing scanning data to mathematicians involved in theoretical and applied optimization—discover useful related algorithms in areas outside of their expertise.

Iterative Optimization in Inverse Problems brings together a number of important iterative algorithms for medical imaging, optimization, and statistical estimation. It incorporates recent work that has not appeared in other books and draws on the author's considerable research in the field, including his recently developed class of SUMMA algorithms

This volume presents a unified approach to constructing iterative methods for solving irregular operator equations and provides rigorous theoretical analysis for several classes of these methods. The analysis of methods includes convergence theorems as well as necessary and sufficient conditions for their convergence at a given rate. The principal groups of methods studied in the book are iterative processes based on the technique of universal linear approximations, stable gradient-type processes, and methods of stable continuous approximations. Compared to existing monographs and textbooks on ill-posed problems, the main distinguishing feature of the presented approach is that it doesn’t require any structural conditions on equations under consideration, except for standard smoothness conditions. This allows to obtain in a uniform style stable iterative methods applicable to wide classes of nonlinear inverse problems. Practical efficiency of suggested algorithms is illustrated in application to inverse problems of potential theory and acoustic scattering. The volume can be read by anyone with a basic knowledge of functional analysis. The book will be of interest to applied mathematicians and specialists in mathematical modeling and inverse problems.

"Fixed-Point Algorithms for Inverse Problems in Science and Engineering" presents some of the most recent work from top-notch researchers studying projection and other first-order fixed-point algorithms in several areas of mathematics and the applied sciences. The material presented provides a survey of the state-of-the-art theory and practice in fixed-point algorithms, identifying emerging problems driven by applications, and discussing new approaches for solving these problems. This book incorporates diverse perspectives from broad-ranging areas of research including, variational analysis, numerical linear algebra, biotechnology, materials science, computational solid-state physics, and chemistry. Topics presented include: Theory of Fixed-point algorithms: convex analysis, convex optimization, subdifferential calculus, nonsmooth analysis, proximal point methods, projection methods, resolvent and related fixed-point theoretic methods, and monotone operator theory. Numerical analysis of fixed-point algorithms: choice of step lengths, of weights, of blocks for block-iterative and parallel methods, and of relaxation parameters; regularization of ill-posed problems; numerical comparison of various methods. Areas of Applications: engineering (image and signal reconstruction and decompression problems), computer tomography and radiation treatment planning (convex feasibility problems), astronomy (adaptive optics), crystallography (molecular structure reconstruction), computational chemistry (molecular structure simulation) and other areas. Because of the variety of applications presented, this book can easily serve as a basis for new and innovated research and collaboration.

The Handbook of Mathematical Methods in Imaging provides a comprehensive treatment of the mathematical techniques used in imaging science. The material is grouped into two central themes, namely, Inverse Problems (Algorithmic Reconstruction) and Signal and Image Processing. Each section within the themes covers applications (modeling), mathematics, numerical methods (using a case example) and open questions. Written by experts in the area, the presentation is mathematically rigorous. The entries are cross-referenced for easy navigation through connected topics. Available in both print and electronic forms, the handbook is enhanced by more than 150 illustrations and an extended bibliography. It will benefit students, scientists and researchers in applied mathematics. Engineers and computer scientists working in imaging will also find this handbook useful.

Inverse problems need to be solved in order to properly interpret indirect measurements. Often, inverse problems are ill-posed and sensitive to data errors. Therefore one has to incorporate some sort of regularization to reconstruct significant information from the given data. A Taste of Inverse Problems: Basic Theory and Examples?presents the main achievements that have emerged in regularization theory over the past 50 years, focusing on linear ill-posed problems and the development of methods that can be applied to them. Some of this material has previously appeared only in journal articles. This book rigorously discusses state-of-the-art inverse problems theory, focusing on numerically relevant aspects and omitting subordinate generalizations; presents diverse real-world applications, important test cases, and possible pitfalls; and treats these applications with the same rigor and depth as the theory.

The field of image reconstruction and inverse problems in imaging have been revolutionized by the introduction of methods which learn to solve inverse problems. This thesis investigates a variety of methods for learning to solve inverse problems by leveraging data: first by exploring the online sparse linear bandit setting, and then by investigating modern methods for leveraging training data to learn to solve inverse problems. In addition, this thesis explores a multi-model method of leveraging human descriptions of change in time series of images to regularize a graph-cut-based change-point detection method. Recent research into learning to solve inverse problems has been dominated by "unrolled optimization" approaches, which unroll a fixed number of iterations of an iterative optimization algorithm, replacing one or more elements of that algorithm with a neural network. These methods have several attractive properties: they can leverage even limited training data to learn accurate reconstructions, they tend to have lower runtime and require fewer iterations than more standard methods which leverage non-learned regularizers, and they are simple to implement and understand. However, learned iterative methods, like most learned inverse problem solvers, are sensitive to small changes in the data measurement model; they are uninterpretable, suffering reduced reconstruction quality if run for more or fewer iterations than were used at train time; and they are limited by memory and numerical constraints to small numbers of iterations, potentially lowering the ceiling for best available reconstruction quality using these methods. This thesis proposes an alternative architecture design based on a Neumann series, which is attractive from a practical perspective for its sample complexity performance and ease to train compared to methods based on unrolled iterative optimization. In addition, this thesis proposes and tests two techniques to adapt arbitrary trained inverse problem solvers to different measurement models, enabling deployment of a single learned model on a variety of forward models without sacrificing performance or requiring potentially-costly new data. Finally, this thesis demonstrates how to train iterative solvers that are unrolled for an arbitrary number of iterations. The proposed technique for the first time permits deep iterative solvers that admit practical convergence guarantees, while allowing flexibility in trading off computation for performance.

Ill-posed problems are encountered in countless areas of real world science and technology. A variety of processes in science and engineering is commonly modeled by algebraic, differential, integral and other equations. In a more difficult case, it can be systems of equations combined with the associated initial and boundary conditions. Frequently, the study of applied optimization problems is also reduced to solving the corresponding equations. These equations, encountered both in theoretical and applied areas, may naturally be classified as operator equations. The current textbook will focus on iterative methods for operator equations in Hilbert spaces.

Inverse problems are concerned with determining causes for observed or desired effects. Problems of this type appear in many application fields both in science and in engineering. The mathematical modelling of inverse problems usually leads to ill-posed problems, i.e., problems where solutions need not exist, need not be unique or may depend discontinuously on the data. For this reason, numerical methods for solving inverse problems are especially difficult, special methods have to be developed which are known under the term "regularization methods". This volume contains twelve survey papers about solution methods for inverse and ill-posed problems and about their application to specific types of inverse problems, e.g., in scattering theory, in tomography and medical applications, in geophysics and in image processing. The papers have been written by leading experts in the field and provide an up-to-date account of solution methods for inverse problems.