Learning Robust Data Driven Methods For Inverse Problems And Change Detection

The field of image reconstruction and inverse problems in imaging have been revolutionized by the introduction of methods which learn to solve inverse problems. This thesis investigates a variety of methods for learning to solve inverse problems by leveraging data: first by exploring the online sparse linear bandit setting, and then by investigating modern methods for leveraging training data to learn to solve inverse problems. In addition, this thesis explores a multi-model method of leveraging human descriptions of change in time series of images to regularize a graph-cut-based change-point detection method. Recent research into learning to solve inverse problems has been dominated by "unrolled optimization" approaches, which unroll a fixed number of iterations of an iterative optimization algorithm, replacing one or more elements of that algorithm with a neural network. These methods have several attractive properties: they can leverage even limited training data to learn accurate reconstructions, they tend to have lower runtime and require fewer iterations than more standard methods which leverage non-learned regularizers, and they are simple to implement and understand. However, learned iterative methods, like most learned inverse problem solvers, are sensitive to small changes in the data measurement model; they are uninterpretable, suffering reduced reconstruction quality if run for more or fewer iterations than were used at train time; and they are limited by memory and numerical constraints to small numbers of iterations, potentially lowering the ceiling for best available reconstruction quality using these methods. This thesis proposes an alternative architecture design based on a Neumann series, which is attractive from a practical perspective for its sample complexity performance and ease to train compared to methods based on unrolled iterative optimization. In addition, this thesis proposes and tests two techniques to adapt arbitrary trained inverse problem solvers to different measurement models, enabling deployment of a single learned model on a variety of forward models without sacrificing performance or requiring potentially-costly new data. Finally, this thesis demonstrates how to train iterative solvers that are unrolled for an arbitrary number of iterations. The proposed technique for the first time permits deep iterative solvers that admit practical convergence guarantees, while allowing flexibility in trading off computation for performance.

The Intelligence Community Studies Board (ICSB) of the National Academies of Sciences, Engineering, and Medicine convened a workshop on December 11â€"12, 2018, in Berkeley, California, to discuss robust machine learning algorithms and systems for the detection and mitigation of adversarial attacks and anomalies. This publication summarizes the presentations and discussions from the workshop.

Anomaly detection is an essential step in the task of automating complex processes, allowing the control algorithm to identify any undesirable operation and take preventive or corrective actions as needed. Even under normal conditions, dynamical systems may have several regimes of operation, and identification of the current operational regime is essential for effective monitoring and control of the process. The most common form of information obtained from these processes is time-series data, which are typically from sensors of various kinds distributed over the system. Thus, many data-driven methods exist for learning how to identify (possible) anomalies from time-series data and to classify the time-series into one of the several classes, including anomalous and normal ones. This dissertation presents one such data-driven time-series analysis method that combines the concepts of Symbolic Time Series Analysis (STSA) and Probabilistic Finite State Automata (PFSA), leading to the concept of $D$-Markov models. This method has several distinct advantages over several state-of-the-art methods, such as; fast execution due to simple algebraic construction, high detection and classification accuracy, and, feature interpretability. Originally developed about two decades ago, the traditional formulation had several shortcomings, which have been largely improved upon in the work reported in this dissertation; including development of a more robust mathematical methodology and augmentation of the formulation to retain more information from the original signal, which allows superior classification performance. This improved PFSA methodology is demonstrated on several real-life engineering problems, as well as some numerical examples, to demonstrate the efficacy of the method in identifying exceptionally complex (e.g., chaotic) signals. The engineering problems include identifying thermo-acoustic instability in combustion systems; detection of crack appearance in structural members, and identification of operational regimes in natural circulation loops. Further, using numerical examples generated from chaotic systems, the PFSA methods are demonstrated to have good classification accuracy and phase change detection capabilities. This dissertation also introduces two novel PFSA-based algorithms for transfer learning and online pattern learning. Transfer learning is a method to learn from one data-set and applying the learnt knowledge to a different, but somewhat similar, data-set without any major re-learning. With many possible applications, the development of a PFSA-based methodology which is computationally faster than the traditional deep learning methods, is an interesting alternative. In the context of online learning, the PFSA-based algorithm is capable of identifying new, previously unseen patterns in real-time while intelligently learning to group the newly observed patterns, thereby expanding its library of classes. All the improvements and new approaches, developed in this dissertation, hope to bring PFSA-based methods to the more mainstream literature and gain widespread usage.

Trains researchers and graduate students in state-of-the-art statistical and machine learning methods to build models with real-world data.

This book examines recent methods for data-driven fault diagnosis of multimode continuous processes. It formalizes, generalizes, and systematically presents the main concepts, and approaches required to design fault diagnosis methods for multimode continuous processes. The book provides both theoretical and practical tools to help readers address the fault diagnosis problem by drawing data-driven methods from at least three different areas: statistics, unsupervised, and supervised learning.

This book is devoted to a special class of engineering problems called Bayesian inverse problems. These problems comprise not only the probabilistic Bayesian formulation of engineering problems, but also the associated stochastic simulation methods needed to solve them. Through this book, the reader will learn how this class of methods can be useful to rigorously address a range of engineering problems where empirical data and fundamental knowledge come into play. The book is written for a non-expert audience and it is contributed to by many of the most renowned academic experts in this field.

Non destructive testing aimed at monitoring, structural identification and di- nostics is of strategic importance in many branches of civil and mechanical - gineering. This type of tests is widely practiced and directly affects topical issues regarding the design of new buildings and the repair and monitoring of existing ones. The load bearing capacity of a structure can now be evaluated using well established mechanical modelling methods aided by computing facilities of great capability. However, to ensure reliable results, models must be calibrated with - curate information on the characteristics of materials and structural components. To this end, non destructive techniques are a useful tool from several points of view. Particularly, by measuring structural response, they provide guidance on the validation of structural descriptions or of the mathematical models of material behaviour. Diagnostic engineering is a crucial area for the application of non destructive testing methods. Repeated tests over time can indicate the emergence of p- sible damage occurring during the structure's lifetime and provide quantitative estimates of the level of residual safety.

Lists citations with abstracts for aerospace related reports obtained from world wide sources and announces documents that have recently been entered into the NASA Scientific and Technical Information Database.

The modern structural health monitoring (SHM) paradigm of transforming in situ, real-time data acquisition into actionable decisions regarding structural performance, health state, maintenance, or life cycle assessment has been accelerated by the rapid growth of “big data” availability and advanced data science. Such data availability coupled with a wide variety of machine learning and data analytics techniques have led to rapid advancement of how SHM is executed, enabling increased transformation from research to practice. This book intends to present a representative collection of such data science advancements used for SHM applications, providing an important contribution for civil engineers, researchers, and practitioners around the world.