Vision Aided Navigation For Autonomous Vehicles Using Tracked Feature Points

This thesis discusses the evaluation, implementation, and testing of several navigation algorithms and feature extraction algorithms using an inertial measurement unit (IMU) and an image capture device (camera) mounted on a ground robot and a quadrotor UAV. The vision-aided navigation algorithms are implemented on data-collected from sensors on an unmanned ground vehicle and a quadrotor, and the results are validated by comparison with GPS data. The thesis investigates sensor fusion techniques for integrating measured IMU data with information extracted from image processing algorithms in order to provide accurate vehicle state estimation. This image-based information takes the forms of features, such as corners, that are tracked over multiple image frames. An extended Kalman filter (EKF) in implemented to fuse vision and IMU data. The main goal of the work is to provide navigation of mobile robots in GPS-denied environments such as indoor environments, cluttered urban environments, or space environments such as asteroids, other planets or the moon. The experimental results show that combining pose information extracted from IMU readings along with pose information extracted from a vision-based algorithm managed to solve the drift problem that comes from using IMU alone and the scale problem that comes from using a monocular vision-based algorithm alone.

This thesis presents vision-based state estimation algorithms for autonomous vehicles to navigate within GPS-denied environments. To accomplish this objective, an approach is developed that utilizes a priori information about the environment. In particular, the algorithm leverages recognizable 'landmarks' in the environment, the positions of which are known in advance, to stabilize the state estimate. Measurements of the position of one or more landmarks in the image plane of a monocular camera are then filtered using an extended Kalman filter (EKF) with data from a traditional inertial measurement unit (IMU) consisting of accelerometers and rate gyros to produce the state estimate. Additionally, the EKF algorithm is adapted to accommodate a stereo camera configuration to measure the distance to a landmark using parallax. The performances of the state estimation algorithms for both the monocular and stereo camera configurations are tested and compared using simulation studies with a quadcopter UAV model. State estimation results are then presented using flight data from a quadcopter UAV instrumented with an IMU and a GoPro camera. It is shown that the proposed landmark navigation method is capable of preventing IMU drift errors by providing a GPS-like measurement when landmarks can be identified. Additionally, the landmark method pairs well with non a priori measurements for interims when landmarks are not available.

This thesis investigate the development of vision-aided navigation algorithms that utilize processed images from a monocular camera as an alternative to GPS. The vision-aided navigation approach explored in this thesis entails defining a set of inertial landmarks, the locations of which are known within the environment, and employing image processing algorithms to detect these landmarks in image frames collected from an onboard monocular camera. These vision-based landmark measurements effectively serve as surrogate GPS measurements that can be incorporated into a navigation filter. Several image processing algorithms were considered for landmark detection and this thesis focuses in particular on two approaches: the continuous adaptive mean shift (CAMSHIFT) algorithm and the adaptable compressive (ADCOM) tracking algorithm. These algorithms are discussed in detail and applied for the detection and tracking of landmarks in monocular camera images. Navigation filters are then designed that employ sensor fusion of accelerometer and rate gyro data from an inertial measurement unit (IMU) with vision-based measurements of the centroids of one or more landmarks in the scene. These filters are tested in simulated navigation scenarios subject to varying levels of sensor and measurement noise and varying number of landmarks. Finally, conclusions and recommendations are provided regarding the implementation of this vision-aided navigation approach for autonomous vehicle navigation systems.

This book constitutes the thoroughly refereed proceedings of the First International Workshop of Computational Neuroscience, held in Porto Allegre, Brazil, in November 2017.The 12 full papers and 3 short papers presented have been thoroughly reviewed andselected from 40 submissions. The papers are organized in topical sections: neural networks; artificial intelligence; computer vision; machine learning; graphic systems and interfaces; decision trees; nonlinear equations; nano-electromechanical systems.

A unified view of the use of computer vision technology for different types of vehicles Computer Vision in Vehicle Technology focuses on computer vision as on-board technology, bringing together fields of research where computer vision is progressively penetrating: the automotive sector, unmanned aerial and underwater vehicles. It also serves as a reference for researchers of current developments and challenges in areas of the application of computer vision, involving vehicles such as advanced driver assistance (pedestrian detection, lane departure warning, traffic sign recognition), autonomous driving and robot navigation (with visual simultaneous localization and mapping) or unmanned aerial vehicles (obstacle avoidance, landscape classification and mapping, fire risk assessment). The overall role of computer vision for the navigation of different vehicles, as well as technology to address on-board applications, is analysed. Key features: Presents the latest advances in the field of computer vision and vehicle technologies in a highly informative and understandable way, including the basic mathematics for each problem. Provides a comprehensive summary of the state of the art computer vision techniques in vehicles from the navigation and the addressable applications points of view. Offers a detailed description of the open challenges and business opportunities for the immediate future in the field of vision based vehicle technologies. This is essential reading for computer vision researchers, as well as engineers working in vehicle technologies, and students of computer vision.

This book constitutes the refereed proceedings of the 4th International Symposium on Ubiquitous Networking, UNet 2018, held in Hammamet, Morocco, in May 2018. The 35 full papers presented together with 5 short papers in this volume were carefully reviewed and selected from 87 submissions. The focus of UNet is on technical challenges and solutions related to such a widespread adoption of networking technologies, including broadband multimedia, machine-to-machine applications, Internet of things, security and privacy, data engineering, sensor networks and RFID technologies.

There is a growing social interest in developing vision-based vehicle guidance systems for improving traffic safety and efficiency and the environment. Ex amples of vision-based vehicle guidance systems include collision warning systems, steering control systems for tracking painted lane marks, and speed control systems for preventing rear-end collisions. Like other guidance systems for aircraft and trains, these systems are ex pected to increase traffic safety significantly. For example, safety improve ments of aircraft landing processes after the introduction of automatic guidance systems have been reported to be 100 times better than prior to installment. Although the safety of human lives is beyond price, the cost for automatic guidance could be compensated by decreased insurance costs. It is becoming more important to increase traffic safety by decreasing the human driver's load in our society, especially with an increasing population of senior people who continue to drive. The second potential social benefit is the improvement of traffic efficiency by decreasing the spacing between vehicles without sacrificing safety. It is reported, for example, that four times the efficiency is expected if the spacing between cars is controlled automatically at 90 cm with a speed of 100 kmjh compared to today's typical manual driving. Although there are a lot of tech nical, psychological, and social issues to be solved before realizing the high density jhigh-speed traffic systems described here, highly efficient highways are becoming more important because of increasing traffic congestion.

The two volume set LNCS 4291 and LNCS 4292 constitutes the refereed proceedings of the Second International Symposium on Visual Computing, ISVC 2006, held in Lake Tahoe, NV, USA in November 2006. The 65 revised full papers and 56 poster papers presented together with 57 papers of ten special tracks were carefully reviewed and selected from more than 280 submissions. The papers cover the four main areas of visual computing.

This monograph is devoted to the theory and development of autonomous navigation of mobile robots using computer vision based sensing mechanism. The conventional robot navigation systems, utilizing traditional sensors like ultrasonic, IR, GPS, laser sensors etc., suffer several drawbacks related to either the physical limitations of the sensor or incur high cost. Vision sensing has emerged as a popular alternative where cameras can be used to reduce the overall cost, maintaining high degree of intelligence, flexibility and robustness. This book includes a detailed description of several new approaches for real life vision based autonomous navigation algorithms and SLAM. It presents the concept of how subgoal based goal-driven navigation can be carried out using vision sensing. The development concept of vision based robots for path/line tracking using fuzzy logic is presented, as well as how a low-cost robot can be indigenously developed in the laboratory with microcontroller based sensor systems. The book describes successful implementation of integration of low-cost, external peripherals, with off-the-shelf procured robots. An important highlight of the book is that it presents a detailed, step-by-step sample demonstration of how vision-based navigation modules can be actually implemented in real life, under 32-bit Windows environment. The book also discusses the concept of implementing vision based SLAM employing a two camera based system.

Autonomous navigation has rapidly grown to become a predominant field of study utilizing recent advances in robotics and artificial intelligence. Most autonomous navigation methods rely on expensive and complex sensor arrays such as Lidar, which pose practical limitations on the widespread deployment of these devices. This thesis presents an end-to-end implementation of a vision-based navigation pipeline for autonomous navigation in pedestrian environments, utilizing only a single front-facing RGB-D camera and tracking camera as perception devices. This pipeline utilizes 3D monocular object tracking in combination with an advanced Kalman-filter based geometric tracking scheme to track nearby pedestrians, in combination with full SLAM for localization and a reinforcement-learning based navigation stack to navigate through challenging dynamic multi-agent environments. The functionality of this pipeline is demonstrated through a series of pedestrian tracking and navigation experiments with many pedestrians. The tracking module of this pipeline is able to correctly localize pedestrians within 0.4 meters in simple scenarios and 0.6 meters in challenging multi-pedestrian stress testing cases inside of a 12 meter space in spite of limited field of view and relying on only inexpensive RGB camera images. Full end-to-end navigation was demonstrated in a crowded environment with 5 pedestrians, with only one collision out of 13 trials.