Computational Photography

Computational photography refers broadly to imaging techniques that enhance or extend the capabilities of digital photography. This new and rapidly developing research field has evolved from computer vision, image processing, computer graphics and applied optics—and numerous commercial products capitalizing on its principles have already appeared in diverse market applications, due to the gradual migration of computational algorithms from computers to imaging devices and software. Computational Photography: Methods and Applications provides a strong, fundamental understanding of theory and methods, and a foundation upon which to build solutions for many of today's most interesting and challenging computational imaging problems. Elucidating cutting-edge advances and applications in digital imaging, camera image processing, and computational photography, with a focus on related research challenges, this book: Describes single capture image fusion technology for consumer digital cameras Discusses the steps in a camera image processing pipeline, such as visual data compression, color correction and enhancement, denoising, demosaicking, super-resolution reconstruction, deblurring, and high dynamic range imaging Covers shadow detection for surveillance applications, camera-driven document rectification, bilateral filtering and its applications, and painterly rendering of digital images Presents machine-learning methods for automatic image colorization and digital face beautification Explores light field acquisition and processing, space-time light field rendering, and dynamic view synthesis with an array of cameras Because of the urgent challenges associated with emerging digital camera applications, image processing methods for computational photography are of paramount importance to research and development in the imaging community. Presenting the work of leading experts, and edited by a renowned authority in digital color imaging and camera image processing, this book considers the rapid developments in this area and addresses very particular research and application problems. It is ideal as a stand-alone professional reference for design and implementation of digital image and video processing tasks, and it can also be used to support graduate courses in computer vision, digital imaging, visual data processing, and computer graphics, among others.

A comprehensive and up-to-date textbook and reference for computational imaging, which combines vision, graphics, signal processing, and optics. Computational imaging involves the joint design of imaging hardware and computer algorithms to create novel imaging systems with unprecedented capabilities. In recent years such capabilities include cameras that operate at a trillion frames per second, microscopes that can see small viruses long thought to be optically irresolvable, and telescopes that capture images of black holes. This text offers a comprehensive and up-to-date introduction to this rapidly growing field, a convergence of vision, graphics, signal processing, and optics. It can be used as an instructional resource for computer imaging courses and as a reference for professionals. It covers the fundamentals of the field, current research and applications, and light transport techniques. The text first presents an imaging toolkit, including optics, image sensors, and illumination, and a computational toolkit, introducing modeling, mathematical tools, model-based inversion, data-driven inversion techniques, and hybrid inversion techniques. It then examines different modalities of light, focusing on the plenoptic function, which describes degrees of freedom of a light ray. Finally, the text outlines light transport techniques, describing imaging systems that obtain micron-scale 3D shape or optimize for noise-free imaging, optical computing, and non-line-of-sight imaging. Throughout, it discusses the use of computational imaging methods in a range of application areas, including smart phone photography, autonomous driving, and medical imaging. End-of-chapter exercises help put the material in context.

Computational Photography combines plentiful computing, digital sensors, modern optics, actuators, probes, and smart lights to escape the limitations of traditional film cameras and enables novel imaging applications. This book provides a practical guide to topics in image capture and manipulation methods for generating compelling pictures for graphics, special effects, scene comprehension, and art. The computational techniques discussed cover topics in exploiting new ideas in manipulating optics, illumination, and sensors at time of capture. In addition, the authors describe sophisticated reconstruction procedures from direct and indirect pixel measurements that go well beyond the traditional digital darkroom experience.

Non-photorealistic rendering (NPR) is a combination of computer graphics and computer vision that produces renderings in various artistic, expressive or stylized ways such as painting and drawing. This book focuses on image and video based NPR, where the input is a 2D photograph or a video rather than a 3D model. 2D NPR techniques have application in areas as diverse as consumer and professional digital photography and visual effects for TV and film production. The book covers the full range of the state of the art of NPR with every chapter authored by internationally renowned experts in the field, covering both classical and contemporary techniques. It will enable both graduate students in computer graphics, computer vision or image processing and professional developers alike to quickly become familiar with contemporary techniques, enabling them to apply 2D NPR algorithms in their own projects.

Over the last decade, digital imaging has become ubiquitous. The advent of cheap digital cameras, and the inclusion of cameras in almost all mobile devices, has made photography one of the basic ways in which people record and communicate experiences. The ubiquity of cameras has imposed new constraints on their physical form. Camera modules are expected to be thin, light, and cheap. These restrictions make the production of high-quality images challenging. We turn to increasingly sophisticated algorithmic tools to transform the raw data captured by a camera into a photograph. This dissertation focuses on one such family of algorithmic tools: those expressible as a Gauss transform. One popular technique in this family is the bilateral filter, which smooths the fine detail in an image without crossing strong edges. It can be used to isolate and control the sharpness, tone, and contrast of a photograph at various scales. Its relatives, the joint-bilateral filter and the joint-bilateral upsample, allow for the fusion of data from multiple images. Another popular technique in the same family is non-local means, which denoises an image by replacing each pixel with the average color of all other pixels in the image with a similar local neighborhood. A naive implementation of these algorithms is prohibitively slow. This dissertation unifies these algorithms under a common framework, describes a variety of applications of the transform in photographic image processing, and presents two new data structures to accelerate the computation of such transforms: the permutohedral lattice, and the Gaussian kd-tree.

The introduction and recent advancements of computational photography have revolutionized the imaging industry. Computational photography is a combination of imaging techniques at the intersection of various fields such as optics, computer vision, and computer graphics. These methods enhance the capabilities of traditional digital photography by applying computational techniques both during and after the capturing process. This thesis targets two major subjects in this field: High Dynamic Range (HDR) image reconstruction and Light Field (LF) compressive capturing, compression, and real-time rendering. The first part of the thesis focuses on the HDR images that concurrently contain detailed information from the very dark shadows to the brightest areas in the scenes. One of the main contributions presented in this thesis is the development of a unified reconstruction algorithm for spatially variant exposures in a single image. This method is based on a camera noise model, and it simultaneously resamples, reconstructs, denoises, and demosaics the image while extending its dynamic range. Furthermore, the HDR reconstruction algorithm is extended to adapt to the local features of the image, as well as the noise statistics, to preserve the high-frequency edges during reconstruction. In the second part of this thesis, the research focus shifts to the acquisition, encoding, reconstruction, and rendering of light field images and videos in a real-time setting. Unlike traditional integral photography, a light field captures the information of the dynamic environment from all angles, all points in space, and all spectral wavelength and time. This thesis employs sparse representation to provide an end-to-end solution to the problem of encoding, real-time reconstruction, and rendering of high dimensional light field video data sets. These solutions are applied on various types of data sets, such as light fields captured with multi-camera systems or hand-held cameras equipped with micro-lens arrays, and spherical light fields. Finally, sparse representation of light fields was utilized for developing a single sensor light field video camera equipped with a color-coded mask. A new compressive sensing model is presented that is suitable for dynamic scenes with temporal coherency and is capable of reconstructing high-resolution light field videos.

This book traces progress in photography since the first pinhole, or camera obscura, architecture. The authors describe innovations such as photogrammetry, and omnidirectional vision for robotic navigation. The text shows how new camera architectures create a need to master related projective geometries for calibration, binocular stereo, static or dynamic scene understanding. Written by leading researchers in the field, this book also explores applications of alternative camera architectures.

Most cameras are inherently designed to mimic what is seen by the human eye: they have three channels of RGB and can achieve up to around 30 frames per second (FPS). However, some cameras are designed to capture other modalities: some may have the ability to capture spectra from near UV to near IR rather than RGB, polarimetry, different times of light travel, etc. Such modalities are as yet unknown, but they can also collect robust data of the scene they are capturing. This book will focus on the emerging computer vision techniques known as computational imaging. These include capturing, processing and analyzing such modalities for various applications of scene understanding.

Image Processing and Acquisition using Python provides readers with a sound foundation in both image acquisition and image processing—one of the first books to integrate these topics together. By improving readers’ knowledge of image acquisition techniques and corresponding image processing, the book will help them perform experiments more effectively and cost efficiently as well as analyze and measure more accurately. Long recognized as one of the easiest languages for non-programmers to learn, Python is used in a variety of practical examples. A refresher for more experienced readers, the first part of the book presents an introduction to Python, Python modules, reading and writing images using Python, and an introduction to images. The second part discusses the basics of image processing, including pre/post processing using filters, segmentation, morphological operations, and measurements. The second part describes image acquisition using various modalities, such as x-ray, CT, MRI, light microscopy, and electron microscopy. These modalities encompass most of the common image acquisition methods currently used by researchers in academia and industry. Features Covers both the physical methods of obtaining images and the analytical processing methods required to understand the science behind the images. Contains many examples, detailed derivations, and working Python examples of the techniques. Offers practical tips on image acquisition and processing. Includes numerous exercises to test the reader’s skills in Python programming and image processing, with solutions to selected problems, example programs, and images available on the book’s web page. New to this edition Machine learning has become an indispensable part of image processing and computer vision, so in this new edition two new chapters are included: one on neural networks and the other on convolutional neural networks. A new chapter on affine transform and many new algorithms. Updated Python code aligned to the latest version of modules.

Covering the theoretical aspects of image processing and analysis through the use of graphs in the representation and analysis of objects, Image Processing and Analysis with Graphs: Theory and Practice also demonstrates how these concepts are indispensible for the design of cutting-edge solutions for real-world applications. Explores new applications in computational photography, image and video processing, computer graphics, recognition, medical and biomedical imaging With the explosive growth in image production, in everything from digital photographs to medical scans, there has been a drastic increase in the number of applications based on digital images. This book explores how graphs—which are suitable to represent any discrete data by modeling neighborhood relationships—have emerged as the perfect unified tool to represent, process, and analyze images. It also explains why graphs are ideal for defining graph-theoretical algorithms that enable the processing of functions, making it possible to draw on the rich literature of combinatorial optimization to produce highly efficient solutions. Some key subjects covered in the book include: Definition of graph-theoretical algorithms that enable denoising and image enhancement Energy minimization and modeling of pixel-labeling problems with graph cuts and Markov Random Fields Image processing with graphs: targeted segmentation, partial differential equations, mathematical morphology, and wavelets Analysis of the similarity between objects with graph matching Adaptation and use of graph-theoretical algorithms for specific imaging applications in computational photography, computer vision, and medical and biomedical imaging Use of graphs has become very influential in computer science and has led to many applications in denoising, enhancement, restoration, and object extraction. Accounting for the wide variety of problems being solved with graphs in image processing and computer vision, this book is a contributed volume of chapters written by renowned experts who address specific techniques or applications. This state-of-the-art overview provides application examples that illustrate practical application of theoretical algorithms. Useful as a support for graduate courses in image processing and computer vision, it is also perfect as a reference for practicing engineers working on development and implementation of image processing and analysis algorithms.