Geometric Methods In Machine Learning And Data Mining

New to the second edition of this advanced text are several chapters on regression, including neural networks and deep learning.

As an introduction to fundamental geometric concepts and tools needed for solving problems of a geometric nature using a computer, this book fills the gap between standard geometry books, which are primarily theoretical, and applied books on computer graphics, computer vision, or robotics that do not cover the underlying geometric concepts in detail. Gallier offers an introduction to affine, projective, computational, and Euclidean geometry, basics of differential geometry and Lie groups, and explores many of the practical applications of geometry. Some of these include computer vision, efficient communication, error correcting codes, cryptography, motion interpolation, and robot kinematics. This comprehensive text covers most of the geometric background needed for conducting research in computer graphics, geometric modeling, computer vision, and robotics and as such will be of interest to a wide audience including computer scientists, mathematicians, and engineers.

Data-driven applications are growing. Machine learning and data analysis now finds both scientific and industrial application in biology, chemistry, geology, medicine, and physics. These applications rely on large quantities of data gathered from automated sensors and user input. Furthermore, the dimensionality of many datasets is extreme: more details are being gathered about single user interactions or sensor readings. All of these applications encounter problems with a common theme: use observed data to make inferences about the world. Our work obtains the first provably efficient algorithms for Independent Component Analysis (ICA) in the presence of heavy-tailed data. The main tool in this result is the centroid body (a well-known topic in convex geometry), along with optimization and random walks for sampling from a convex body. This is the first algorithmic use of the centroid body and it is of independent theoretical interest, since it effectively replaces the estimation of covariance from samples, and is more generally accessible. We demonstrate that ICA is itself a powerful geometric primitive. That is, having access to an efficient algorithm for ICA enables us to efficiently solve other important problems in machine learning. The first such reduction is a solution to the open problem of efficiently learning the intersection of n + 1 halfspaces in Rn, posed in [43]. This reduction relies on a non-linear transformation of samples from such an intersection of halfspaces (i.e. a simplex) to samples which are approximately from a linearly transformed product distribution. Through this transformation of samples, which can be done efficiently, one can then use an ICA algorithm to recover the vertices of the intersection of halfspaces. Finally, we again use ICA as an algorithmic primitive to construct an efficient solution to the widely-studied problem of learning the parameters of a Gaussian mixture model. Our algorithm again transforms samples from a Gaussian mixture model into samples which fit into the ICA model and, when processed by an ICA algorithm, result in recovery of the mixture parameters. Our algorithm is effective even when the number of Gaussians in the mixture grows with the ambient dimension, even polynomially in the dimension. In addition to the efficient parameter estimation, we also obtain a complexity lower bound for a low-dimension Gaussian mixture model.

The focus of this book is on providing students with insights into geometry that can help them understand deep learning from a unified perspective. Rather than describing deep learning as an implementation technique, as is usually the case in many existing deep learning books, here, deep learning is explained as an ultimate form of signal processing techniques that can be imagined. To support this claim, an overview of classical kernel machine learning approaches is presented, and their advantages and limitations are explained. Following a detailed explanation of the basic building blocks of deep neural networks from a biological and algorithmic point of view, the latest tools such as attention, normalization, Transformer, BERT, GPT-3, and others are described. Here, too, the focus is on the fact that in these heuristic approaches, there is an important, beautiful geometric structure behind the intuition that enables a systematic understanding. A unified geometric analysis to understand the working mechanism of deep learning from high-dimensional geometry is offered. Then, different forms of generative models like GAN, VAE, normalizing flows, optimal transport, and so on are described from a unified geometric perspective, showing that they actually come from statistical distance-minimization problems. Because this book contains up-to-date information from both a practical and theoretical point of view, it can be used as an advanced deep learning textbook in universities or as a reference source for researchers interested in acquiring the latest deep learning algorithms and their underlying principles. In addition, the book has been prepared for a codeshare course for both engineering and mathematics students, thus much of the content is interdisciplinary and will appeal to students from both disciplines.

A comprehensive overview of data mining from an algorithmic perspective, integrating related concepts from machine learning and statistics.

The field of computational learning theory arose out of the desire to for mally understand the process of learning. As potential applications to artificial intelligence became apparent, the new field grew rapidly. The learning of geo metric objects became a natural area of study. The possibility of using learning techniques to compensate for unsolvability provided an attraction for individ uals with an immediate need to solve such difficult problems. Researchers at the Center for Night Vision were interested in solving the problem of interpreting data produced by a variety of sensors. Current vision techniques, which have a strong geometric component, can be used to extract features. However, these techniques fall short of useful recognition of the sensed objects. One potential solution is to incorporate learning techniques into the geometric manipulation of sensor data. As a first step toward realizing such a solution, the Systems Research Center at the University of Maryland, in conjunction with the Center for Night Vision, hosted a Workshop on Learning and Geometry in January of 1991. Scholars in both fields came together to learn about each others' field and to look for common ground, with the ultimate goal of providing a new model of learning from geometrical examples that would be useful in computer vision. The papers in the volume are a partial record of that meeting.

This is the first comprehensive book on information geometry, written by the founder of the field. It begins with an elementary introduction to dualistic geometry and proceeds to a wide range of applications, covering information science, engineering, and neuroscience. It consists of four parts, which on the whole can be read independently. A manifold with a divergence function is first introduced, leading directly to dualistic structure, the heart of information geometry. This part (Part I) can be apprehended without any knowledge of differential geometry. An intuitive explanation of modern differential geometry then follows in Part II, although the book is for the most part understandable without modern differential geometry. Information geometry of statistical inference, including time series analysis and semiparametric estimation (the Neyman–Scott problem), is demonstrated concisely in Part III. Applications addressed in Part IV include hot current topics in machine learning, signal processing, optimization, and neural networks. The book is interdisciplinary, connecting mathematics, information sciences, physics, and neurosciences, inviting readers to a new world of information and geometry. This book is highly recommended to graduate students and researchers who seek new mathematical methods and tools useful in their own fields.

An up-to-date account of algebraic statistics and information geometry, which also explores the emerging connections between these two disciplines.

Over the past 15 years, there has been a growing need in the medical image computing community for principled methods to process nonlinear geometric data. Riemannian geometry has emerged as one of the most powerful mathematical and computational frameworks for analyzing such data. Riemannian Geometric Statistics in Medical Image Analysis is a complete reference on statistics on Riemannian manifolds and more general nonlinear spaces with applications in medical image analysis. It provides an introduction to the core methodology followed by a presentation of state-of-the-art methods. Beyond medical image computing, the methods described in this book may also apply to other domains such as signal processing, computer vision, geometric deep learning, and other domains where statistics on geometric features appear. As such, the presented core methodology takes its place in the field of geometric statistics, the statistical analysis of data being elements of nonlinear geometric spaces. The foundational material and the advanced techniques presented in the later parts of the book can be useful in domains outside medical imaging and present important applications of geometric statistics methodology Content includes: The foundations of Riemannian geometric methods for statistics on manifolds with emphasis on concepts rather than on proofs Applications of statistics on manifolds and shape spaces in medical image computing Diffeomorphic deformations and their applications As the methods described apply to domains such as signal processing (radar signal processing and brain computer interaction), computer vision (object and face recognition), and other domains where statistics of geometric features appear, this book is suitable for researchers and graduate students in medical imaging, engineering and computer science. A complete reference covering both the foundations and state-of-the-art methods Edited and authored by leading researchers in the field Contains theory, examples, applications, and algorithms Gives an overview of current research challenges and future applications