Vision Aided Navigation For Gps Denied Environments Using Landmark Feature Identification

This thesis investigate the development of vision-aided navigation algorithms that utilize processed images from a monocular camera as an alternative to GPS. The vision-aided navigation approach explored in this thesis entails defining a set of inertial landmarks, the locations of which are known within the environment, and employing image processing algorithms to detect these landmarks in image frames collected from an onboard monocular camera. These vision-based landmark measurements effectively serve as surrogate GPS measurements that can be incorporated into a navigation filter. Several image processing algorithms were considered for landmark detection and this thesis focuses in particular on two approaches: the continuous adaptive mean shift (CAMSHIFT) algorithm and the adaptable compressive (ADCOM) tracking algorithm. These algorithms are discussed in detail and applied for the detection and tracking of landmarks in monocular camera images. Navigation filters are then designed that employ sensor fusion of accelerometer and rate gyro data from an inertial measurement unit (IMU) with vision-based measurements of the centroids of one or more landmarks in the scene. These filters are tested in simulated navigation scenarios subject to varying levels of sensor and measurement noise and varying number of landmarks. Finally, conclusions and recommendations are provided regarding the implementation of this vision-aided navigation approach for autonomous vehicle navigation systems.

This thesis presents vision-based state estimation algorithms for autonomous vehicles to navigate within GPS-denied environments. To accomplish this objective, an approach is developed that utilizes a priori information about the environment. In particular, the algorithm leverages recognizable 'landmarks' in the environment, the positions of which are known in advance, to stabilize the state estimate. Measurements of the position of one or more landmarks in the image plane of a monocular camera are then filtered using an extended Kalman filter (EKF) with data from a traditional inertial measurement unit (IMU) consisting of accelerometers and rate gyros to produce the state estimate. Additionally, the EKF algorithm is adapted to accommodate a stereo camera configuration to measure the distance to a landmark using parallax. The performances of the state estimation algorithms for both the monocular and stereo camera configurations are tested and compared using simulation studies with a quadcopter UAV model. State estimation results are then presented using flight data from a quadcopter UAV instrumented with an IMU and a GoPro camera. It is shown that the proposed landmark navigation method is capable of preventing IMU drift errors by providing a GPS-like measurement when landmarks can be identified. Additionally, the landmark method pairs well with non a priori measurements for interims when landmarks are not available.

This thesis discusses the evaluation, implementation, and testing of several navigation algorithms and feature extraction algorithms using an inertial measurement unit (IMU) and an image capture device (camera) mounted on a ground robot and a quadrotor UAV. The vision-aided navigation algorithms are implemented on data-collected from sensors on an unmanned ground vehicle and a quadrotor, and the results are validated by comparison with GPS data. The thesis investigates sensor fusion techniques for integrating measured IMU data with information extracted from image processing algorithms in order to provide accurate vehicle state estimation. This image-based information takes the forms of features, such as corners, that are tracked over multiple image frames. An extended Kalman filter (EKF) in implemented to fuse vision and IMU data. The main goal of the work is to provide navigation of mobile robots in GPS-denied environments such as indoor environments, cluttered urban environments, or space environments such as asteroids, other planets or the moon. The experimental results show that combining pose information extracted from IMU readings along with pose information extracted from a vision-based algorithm managed to solve the drift problem that comes from using IMU alone and the scale problem that comes from using a monocular vision-based algorithm alone.

Traditionally, the task of determining aircraft position and attitude for automatic control has been handled by the combination of an inertial measurement unit (IMU) with a Global Positioning System (GPS) receiver. In this configuration, accelerations and angular rates from the IMU can be integrated forward in time, and position updates from the GPS can be used to bound the errors that result from this integration. However, reliance on the reception of GPS signals places artificial constraints on aircraft such as small unmanned aerial vehicles (UAVs) that are otherwise physically capable of operation in indoor, cluttered, or adversarial environments. Theefore, this work investigates methods for incorporating a monocular vision sensor into a standard avionics suite. Vision sensors possess the potential to extract information about the surrounding environment and determine the locations of features or points of interest. Having mapped out landmarks in an unknown environment, subsequent observations by the vision sensor can in turn be used to resolve aircraft position and orientation while continuing to map out new features. An extended Kalman filter framework for performing the tasks of vision-based mapping and navigation is presented. Feature points are detected in each image using a Harris corner detector, and these feature measurements are corresponded from frame to frame using a statistical Z-test. When GPS is available, sequential observations of a single landmark point allow the point's location in inertial space to be estimated. When GPS is not available, landmarks that have been sufficiently triangulated can be used for estimating vehicle position and attitude. Simulation and real-time flight test results for vision-based mapping and navigation are presented to demonstrate feasibility in real-time applications. These methods are then integrated into a practical framework for flight in GPS-denied environments and verified through the autonomous flight of a UAV during a loss-of-GPS scenario. The methodology is also extended to the application of vehicles equipped with stereo vision systems. This framework enables aircraft capable of hovering in place to maintain a bounded pose estimate indefinitely without drift during a GPS outage.

Deep learning algorithms have brought a revolution to the computer vision community by introducing non-traditional and efficient solutions to several image-related problems that had long remained unsolved or partially addressed. This book presents a collection of eleven chapters where each individual chapter explains the deep learning principles of a specific topic, introduces reviews of up-to-date techniques, and presents research findings to the computer vision community. The book covers a broad scope of topics in deep learning concepts and applications such as accelerating the convolutional neural network inference on field-programmable gate arrays, fire detection in surveillance applications, face recognition, action and activity recognition, semantic segmentation for autonomous driving, aerial imagery registration, robot vision, tumor detection, and skin lesion segmentation as well as skin melanoma classification. The content of this book has been organized such that each chapter can be read independently from the others. The book is a valuable companion for researchers, for postgraduate and possibly senior undergraduate students who are taking an advanced course in related topics, and for those who are interested in deep learning with applications in computer vision, image processing, and pattern recognition.

Quad Rotorcraft Control develops original control methods for the navigation and hovering flight of an autonomous mini-quad-rotor robotic helicopter. These methods use an imaging system and a combination of inertial and altitude sensors to localize and guide the movement of the unmanned aerial vehicle relative to its immediate environment. The history, classification and applications of UAVs are introduced, followed by a description of modelling techniques for quad-rotors and the experimental platform itself. A control strategy for the improvement of attitude stabilization in quad-rotors is then proposed and tested in real-time experiments. The strategy, based on the use low-cost components and with experimentally-established robustness, avoids drift in the UAV’s angular position by the addition of an internal control loop to each electronic speed controller ensuring that, during hovering flight, all four motors turn at almost the same speed. The quad-rotor’s Euler angles being very close to the origin, other sensors like GPS or image-sensing equipment can be incorporated to perform autonomous positioning or trajectory-tracking tasks. Two vision-based strategies, each designed to deal with a specific kind of mission, are introduced and separately tested. The first stabilizes the quad-rotor over a landing pad on the ground; it extracts the 3-dimensional position using homography estimation and derives translational velocity by optical flow calculation. The second combines colour-extraction and line-detection algorithms to control the quad-rotor’s 3-dimensional position and achieves forward velocity regulation during a road-following task. In order to estimate the translational-dynamical characteristics of the quad-rotor (relative position and translational velocity) as they evolve within a building or other unstructured, GPS-deprived environment, imaging, inertial and altitude sensors are combined in a state observer. The text give the reader a current view of the problems encountered in UAV control, specifically those relating to quad-rotor flying machines and it will interest researchers and graduate students working in that field. The vision-based control strategies presented help the reader to a better understanding of how an imaging system can be used to obtain the information required for performance of the hovering and navigation tasks ubiquitous in rotored UAV operation.

Vision aided navigation has become an important topic of research for assisting with navigational tasks when traditional navigation technology is not functioning properly, such as GPS devices. Approaches based upon traditional visible and thermal camera data have been shown to be effective across a wide range of scenarios; however, there are situations (such as over deserts and large bodies of water) where there is not enough features in the imagery for it to be effective. Hyperspectral imagery (HSI) has not been heavily explored for vision-aided navigation applications and can provide a wealth of features to aid with these tasks. In this work we investigate the use of HSI in vision-aided navigation using machine learning. This work aims to show that HSI can provide utility in vision-aided navigation and can be more effective than conventional imagery approaches for certain tasks. We explore the use of HSI data selected from different geographic locations across the United States to demonstrate that spectral features from these different locales can be used to distinguish and classify them. The HSI data used was collected with NASA's AVIRIS sensor due to the availability of large amounts of data across the United States. The neural network that was trained and tested was a convolutional neural network that uses a multi-scale filter bank and residual learning to improve the network. A second network was created, trained, and tested on equivalent RGB images so that performance could be compared against the HSI network. We tested the networks in two ways. First, we made use of all selected data from each location class that resulted in an unequal number of location samples, and second where we selected an equal number of samples from each location class. We optimized the hyper parameters to yield best performance in each case and found that the best HSI network out performed the best RGB network in every test case. While this work is preliminary, it demonstrates the strong potential of HSI for vision aided navigation applications.

The book presents a comprehensive and up-to-date review of fuzzy pattern recognition. It carefully discusses a range of methodological and algorithmic issues, as well as implementations and case studies, and identifies the best design practices, assesses business models and practices of pattern recognition in real-world applications in industry, health care, administration, and business. Since the inception of fuzzy sets, fuzzy pattern recognition with its methodology, algorithms, and applications, has offered new insights into the principles and practice of pattern classification. Computational intelligence (CI) establishes a comprehensive framework aimed at fostering the paradigm of pattern recognition. The collection of contributions included in this book offers a representative overview of the advances in the area, with timely, in-depth and comprehensive material on the conceptually appealing and practically sound methodology and practices of CI-based pattern recognition.