Perception Driven Optimal Motion Planning Under Resource Constraints

Over the past few years there has been a new wave of interest in fully autonomous robots operating in the real world, with applications from autonomous driving to search and rescue. These robots are expected to operate at high speeds in unknown, unstructured environments using only onboard sensing and computation, presenting significant challenges for high performance autonomous navigation. To enable research in these challenging scenarios, the first part of this thesis focuses on the development of a custom high-performance research UAV capable of high speed autonomous flight using only vision and inertial sensors. This research platform was used to develop stateof-the-art onboard visual inertial state estimation at high speeds in challenging scenarios such as flying through window gaps. While this platform is capable of high performance state estimation and control, its capabilities in unknown environments are severely limited by the computational costs of running traditional vision-based mapping and motion planning algorithms on an embedded platform. Motivated by these challenges, the second part of this thesis presents an algorithmic approach to the problem of motion planning in an unknown environment when the computational costs of mapping all available sensor data is prohibitively high. The algorithm is built around a tree of dynamically feasible and free space optimal trajectories to the goal state in configuration space. As the algorithm progresses it iteratively switches between processing new sensor data and locally updating the search tree. We show that the algorithm produces globally optimal motion plans, matching the optimal solution for the case with the full (unprocessed) sensor data, while only processing a subset of the data. The mapping and motion planning algorithm is demonstrated on a number of test systems, with a particular focus on a six-dimensional thrust limited model of a quadrotor.

This thesis studies algorithmic aspects of the question: "How quickly can a vision-driven MAV traverse a given path, while maintaining accurate state estimates at all times?" We seek tractable families of problems involving designing a time-optimal open-loop sequence of controls for a MAV subject to both actuation as well as perception constraints that allow the robot leverage its onboard camera for accurate state estimation. Prior work has either focused on asymptotically optimal search-based approaches which are challenging to implement in real time, or fast local-optimization-based methods with no guarantees on global constraint satisfaction, stability, or optimality.

During the last decades, motion planning for autonomous systems has become an important area of research. The high interest is not the least due to the development of systems such as self-driving cars, unmanned aerial vehicles and robotic manipulators. The objective in optimal motion planning problems is to find feasible motion plans that also optimize a performance measure. From a control perspective, the problem is an instance of an optimal control problem. This thesis addresses optimal motion planning problems for complex dynamical systems that operate in unstructured environments, where no prior reference such as road-lane information is available. Some example scenarios are autonomous docking of vessels in harbors and autonomous parking of self-driving tractor-trailer vehicles at loading sites. The focus is to develop optimal motion planning algorithms that can reliably be applied to these types of problems. This is achieved by combining recent ideas from automatic control, numerical optimization and robotics. The first contribution is a systematic approach for computing local solutions to motion planning problems in challenging unstructured environments. The solutions are computed by combining homotopy methods and direct optimal control techniques. The general principle is to define a homotopy that transforms, or preferably relaxes, the original problem to an easily solved problem. The approach is demonstrated in motion planning problems in 2D and 3D environments, where the presented method outperforms a state-of-the-art asymptotically optimal motion planner based on random sampling. The second contribution is an optimization-based framework for automatic generation of motion primitives for lattice-based motion planners. Given a family of systems, the user only needs to specify which principle types of motions that are relevant for the considered system family. Based on the selected principle motions and a selected system instance, the framework computes a library of motion primitives by simultaneously optimizing the motions and the terminal states. The final contribution of this thesis is a motion planning framework that combines the strengths of sampling-based planners with direct optimal control in a novel way. The sampling-based planner is applied to the problem in a first step using a discretized search space, where the system dynamics and objective function are chosen to coincide with those used in a second step based on optimal control. This combination ensures that the sampling-based motion planner provides a feasible motion plan which is highly suitable as warm-start to the optimal control step. Furthermore, the second step is modified such that it also can be applied in a receding-horizon fashion, where the proposed combination of methods is used to provide theoretical guarantees in terms of recursive feasibility, worst-case objective function value and convergence to the terminal state. The proposed motion planning framework is successfully applied to several problems in challenging unstructured environments for tractor-trailer vehicles. The framework is also applied and tailored for maritime navigation for vessels in archipelagos and harbors, where it is able to compute energy-efficient trajectories which complies with the international regulations for preventing collisions at sea.

During the last decades, motion planning for autonomous systems has become an important area of research. The high interest is not the least due to the development of systems such as self-driving cars, unmanned aerial vehicles and robotic manipulators. In this thesis, the objective is not only to find feasible solutions to a motion planning problem, but solutions that also optimize some kind of performance measure. From a control perspective, the resulting problem is an instance of an optimal control problem. In this thesis, the focus is to further develop optimal control algorithms such that they be can used to obtain improved solutions to motion planning problems. This is achieved by combining ideas from automatic control, numerical optimization and robotics. First, a systematic approach for computing local solutions to motion planning problems in challenging environments is presented. The solutions are computed by combining homotopy methods and numerical optimal control techniques. The general principle is to define a homotopy that transforms, or preferably relaxes, the original problem to an easily solved problem. The approach is demonstrated in motion planning problems in 2D and 3D environments, where the presented method outperforms both a state-of-the-art numerical optimal control method based on standard initialization strategies and a state-of-the-art optimizing sampling-based planner based on random sampling. Second, a framework for automatically generating motion primitives for lattice-based motion planners is proposed. Given a family of systems, the user only needs to specify which principle types of motions that are relevant for the considered system family. Based on the selected principle motions and a selected system instance, the algorithm not only automatically optimizes the motions connecting pre-defined boundary conditions, but also simultaneously optimizes the terminal state constraints as well. In addition to handling static a priori known system parameters such as platform dimensions, the framework also allows for fast automatic re-optimization of motion primitives if the system parameters change while the system is in use. Furthermore, the proposed framework is extended to also allow for an optimization of discretization parameters, that are are used by the lattice-based motion planner to define a state-space discretization. This enables an optimized selection of these parameters for a specific system instance. Finally, a unified optimization-based path planning approach to efficiently compute locally optimal solutions to advanced path planning problems is presented. The main idea is to combine the strengths of sampling-based path planners and numerical optimal control. The lattice-based path planner is applied to the problem in a first step using a discretized search space, where system dynamics and objective function are chosen to coincide with those used in a second numerical optimal control step. This novel tight combination of a sampling-based path planner and numerical optimal control makes, in a structured way, benefit of the former method’s ability to solve combinatorial parts of the problem and the latter method’s ability to obtain locally optimal solutions not constrained to a discretized search space. The proposed approach is shown in several practically relevant path planning problems to provide improvements in terms of computation time, numerical reliability, and objective function value.

With the rapid development of artificial intelligence and the emergence of various new sensors, autonomous driving has grown in popularity in recent years. The implementation of autonomous driving requires new sources of sensory data, such as cameras, radars, and lidars, and the algorithm processing requires a high degree of parallel computing. In this regard, traditional CPUs have insufficient computing power, while DSPs are good at image processing but lack sufficient performance for deep learning. Although GPUs are good at training, they are too “power-hungry,” which can affect vehicle performance. Therefore, this book looks to the future, arguing that custom ASICs are bound to become mainstream. With the goal of ICs design for autonomous driving, this book discusses the theory and engineering practice of designing future-oriented autonomous driving SoC chips. The content is divided into thirteen chapters, the first chapter mainly introduces readers to the current challenges and research directions in autonomous driving. Chapters 2–6 focus on algorithm design for perception and planning control. Chapters 7–10 address the optimization of deep learning models and the design of deep learning chips, while Chapters 11-12 cover automatic driving software architecture design. Chapter 13 discusses the 5G application on autonomous drving. This book is suitable for all undergraduates, graduate students, and engineering technicians who are interested in autonomous driving.

The first handbook to the practical specifics of motion planning, Optimized-Motion Planning offers design engineers methods and insights for solving real motion planning problems in a 3-dimensional space. Complete with a disk of software programs, this unique guide allows users to design, test, and implement possible solutions, useful in a host of contexts, especially tool path planning. Beginning with a brief overview of the general class of problems examined within the book as well as available solution techniques, Part 1 familiarizes the reader with the conceptual threads that underlie each approach. This early discussion also considers the specific applications of each technique as well as its computational efficiency. Part 2 illustrates basic problem-solving methodology by considering the case of a point moving between stationary polygons in a plane. This section features algorithms for data organization and storage, the concepts of passage networks and feasibility charts, as well as the path optimization algorithm. Elaborating on the problematic model described in Part 2, Part 3 develops an algorithm for optimizing the motion of a point between stationary polyhedra in a 3-dimensional space. This algorithm is first applied to the case of nonpoint objects moving between obstacles that can be stationary or moving with known patterns. It's then used in connection with the extensively investigated problem of motion planning for multilink manipulators.

This book presents the Proceedings from the Second International Workshop GLOW 2024 held in conjunction with the International Joint Conference on Artificial Intelligence, IJCAI 2024, in Jeju Island, South Korea, in August 2024. The 11 full papers and 4 short papers included in this book were carefully reviewed and selected from 22 submissions. They were organized in topical sections as follows: efficient methods for low-resource hardware; efficient fintuning with limited data; advancements in multimodal systems; recognition and reasoning in the open world.

Snake Robots is a novel treatment of theoretical and practical topics related to snake robots: robotic mechanisms designed to move like biological snakes and able to operate in challenging environments in which human presence is either undesirable or impossible. Future applications of such robots include search and rescue, inspection and maintenance, and subsea operations. Locomotion in unstructured environments is a focus for this book. The text targets the disparate muddle of approaches to modelling, development and control of snake robots in current literature, giving a unified presentation of recent research results on snake robot locomotion to increase the reader’s basic understanding of these mechanisms and their motion dynamics and clarify the state of the art in the field. The book is a complete treatment of snake robotics, with topics ranging from mathematical modelling techniques, through mechatronic design and implementation, to control design strategies. The development of two snake robots is described and both are used to provide experimental validation of many of the theoretical results. Snake Robots is written in a clear and easily understandable manner which makes the material accessible by specialists in the field and non-experts alike. Numerous illustrative figures and images help readers to visualize the material. The book is particularly useful to new researchers taking on a topic related to snake robots because it provides an extensive overview of the snake robot literature and also represents a suitable starting point for research in this area.