Automated Configuration Of Algorithms For Solving Hard Computational Problems

The best-performing algorithms for many hard problems are highly parameterized. Selecting the best heuristics and tuning their parameters for optimal overall performance is often a difficult, tedious, and unsatisfying task. This thesis studies the automation of this important part of algorithm design: the configuration of discrete algorithm components and their continuous parameters to construct an algorithm with desirable empirical performance characteristics. Automated configuration procedures can facilitate algorithm development and be applied on the end user side to optimize performance for new instance types and optimization objectives. The use of such procedures separates high-level cognitive tasks carried out by humans from tedious low-level tasks that can be left to machines. We introduce two alternative algorithm configuration frameworks: iterated local search in parameter configuration space and sequential optimization based on response surface models. To the best of our knowledge, our local search approach is the first that goes beyond local optima. Our model-based search techniques significantly outperform existing techniques and extend them in ways crucial for general algorithm configuration: they can handle categorical parameters, optimization objectives defined across multiple instances, and tens of thousands of data points. We study how many runs to perform for evaluating a parameter configuration and how to set the cutoff time, after which algorithm runs are terminated unsuccessfully. We introduce data-driven approaches for making these choices adaptively, most notably the first general method for adaptively setting the cutoff time. Using our procedures—to the best of our knowledge still the only ones applicable to these complex configuration tasks—we configured state-of-the-art tree search and local search algorithms for SAT, as well as CPLEX, the most widely-used commercial optimization tool for solving mixed integer programs (MIP). In many cases,

This open access book presents the first comprehensive overview of general methods in Automated Machine Learning (AutoML), collects descriptions of existing systems based on these methods, and discusses the first series of international challenges of AutoML systems. The recent success of commercial ML applications and the rapid growth of the field has created a high demand for off-the-shelf ML methods that can be used easily and without expert knowledge. However, many of the recent machine learning successes crucially rely on human experts, who manually select appropriate ML architectures (deep learning architectures or more traditional ML workflows) and their hyperparameters. To overcome this problem, the field of AutoML targets a progressive automation of machine learning, based on principles from optimization and machine learning itself. This book serves as a point of entry into this quickly-developing field for researchers and advanced students alike, as well as providing a reference for practitioners aiming to use AutoML in their work.

This book constitutes the refereed proceedings of the 7th International Conference on Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, CPAIOR 2010, held in Bologna, Italy, in June 2010. The 18 revised full papers and 17 revised short papers presented together with the extended abstracts of 3 invited talks were carefully reviewed and selected from 72 submissions. The papers are focused on both theoretical and practical, application-oriented issues and present current research with a special focus on the integration and hybridization of the approaches of constraint programming, artificial intelligence, and operations research technologies for solving large scale and complex real life combinatorial optimization problems.

This book constitutes the refereed proceedings of the 7th International Conference on Theory and Practice of Natural Computing, TPNC 2017, held in Dublin, Ireland, in December 2018. The 35 full papers presented in this book, together with one invited talk, were carefully reviewed and selected from 69 submissions. The papers are organized around the following topical sections: applications of natural computing as algorithms, bioinformatics, control, cryptography, design, economics. The more theoretical contributions handle with artificial chemistry, artificial immune systems, artificial life, cellular automata, cognitive computing, cognitive engineering, cognitive robotics, collective behaviour, complex systems, computational intelligence, computational social science, computing with words, developmental systems, DNA computing, DNA nanotechnology, evolutionary algorithms, evolutionary computing, evolutionary game theory, fractal geometry, fuzzy control, fuzzy logic, fuzzy sets, fuzzy systems, genetic algorithms, genetic programming, granular computing, heuristics, intelligent agents, intelligent systems, machine intelligence, molecular programming, neural computing, neural networks, quantum communication, quantum computing, rough sets, self-assembly.

This book constitutes the thoroughly refereed post-conference proceedings of the 6th International Conference on Learning and Intelligent Optimization, LION 6, held in Paris, France, in January 2012. The 23 long and 30 short revised papers were carefully reviewed and selected from a total of 99 submissions. The papers focus on the intersections and uncharted territories between machine learning, artificial intelligence, mathematical programming and algorithms for hard optimization problems. In addition to the paper contributions the conference also included 3 invited speakers, who presented forefront research results and frontiers, and 3 tutorial talks, which were crucial in bringing together the different components of LION community.

Decades of innovations in combinatorial problem solving have produced better and more complex algorithms. These new methods are better since they can solve larger problems and address new application domains. They are also more complex which means that they are hard to reproduce and often harder to fine-tune to the peculiarities of a given problem. This last point has created a paradox where efficient tools are out of reach of practitioners. Autonomous search (AS) represents a new research field defined to precisely address the above challenge. Its major strength and originality consist in the fact that problem solvers can now perform self-improvement operations based on analysis of the performances of the solving process -- including short-term reactive reconfiguration and long-term improvement through self-analysis of the performance, offline tuning and online control, and adaptive control and supervised control. Autonomous search "crosses the chasm" and provides engineers and practitioners with systems that are able to autonomously self-tune their performance while effectively solving problems. This is the first book dedicated to this topic, and it can be used as a reference for researchers, engineers, and postgraduates in the areas of constraint programming, machine learning, evolutionary computing, and feedback control theory. After the editors' introduction to autonomous search, the chapters are focused on tuning algorithm parameters, autonomous complete (tree-based) constraint solvers, autonomous control in metaheuristics and heuristics, and future autonomous solving paradigms. Autonomous search (AS) represents a new research field defined to precisely address the above challenge. Its major strength and originality consist in the fact that problem solvers can now perform self-improvement operations based on analysis of the performances of the solving process -- including short-term reactive reconfiguration and long-term improvement through self-analysis of the performance, offline tuning and online control, and adaptive control and supervised control. Autonomous search "crosses the chasm" and provides engineers and practitioners with systems that are able to autonomously self-tune their performance while effectively solving problems. This is the first book dedicated to this topic, and it can be used as a reference for researchers, engineers, and postgraduates in the areas of constraint programming, machine learning, evolutionary computing, and feedback control theory. After the editors' introduction to autonomous search, the chapters are focused on tuning algorithm parameters, autonomous complete (tree-based) constraint solvers, autonomous control in metaheuristics and heuristics, and future autonomous solving paradigms. This is the first book dedicated to this topic, and it can be used as a reference for researchers, engineers, and postgraduates in the areas of constraint programming, machine learning, evolutionary computing, and feedback control theory. After the editors' introduction to autonomous search, the chapters are focused on tuning algorithm parameters, autonomous complete (tree-based) constraint solvers, autonomous control in metaheuristics and heuristics, and future autonomous solving paradigms. This is the first book dedicated to this topic, and it can be used as a reference for researchers, engineers, and postgraduates in the areas of constraint programming, machine learning, evolutionary computing, and feedback control theory. After the editors' introduction to autonomous search, the chapters are focused on tuning algorithm parameters, autonomous complete (tree-based) constraint solvers, autonomous control in metaheuristics and heuristics, and future autonomous solving paradigms.

This book constitutes the refereed proceedings of the 13th International Conference on Parallel Problem Solving from Nature, PPSN 2013, held in Ljubljana, Slovenia, in September 2014. The total of 90 revised full papers were carefully reviewed and selected from 217 submissions. The meeting began with 7 workshops which offered an ideal opportunity to explore specific topics in evolutionary computation, bio-inspired computing and metaheuristics. PPSN XIII also included 9 tutorials. The papers are organized in topical sections on adaption, self-adaption and parameter tuning; classifier system, differential evolution and swarm intelligence; coevolution and artificial immune systems; constraint handling; dynamic and uncertain environments; estimation of distribution algorithms and metamodelling; genetic programming; multi-objective optimisation; parallel algorithms and hardware implementations; real world applications; and theory.

This book disseminates and promotes the recent research progress and frontier development on AutoML and meta-learning as well as their applications on computer vision, natural language processing, multimedia and data mining related fields. These are exciting and fast-growing research directions in the general field of machine learning. The authors advocate novel, high-quality research findings, and innovative solutions to the challenging problems in AutoML and meta-learning. This topic is at the core of the scope of artificial intelligence, and is attractive to audience from both academia and industry. This book is highly accessible to the whole machine learning community, including: researchers, students and practitioners who are interested in AutoML, meta-learning, and their applications in multimedia, computer vision, natural language processing and data mining related tasks. The book is self-contained and designed for introductory and intermediate audiences. No special prerequisite knowledge is required to read this book.

Hybrid Optimization focuses on the application of artificial intelligence and operations research techniques to constraint programming for solving combinatorial optimization problems. This book covers the most relevant topics investigated in the last ten years by leading experts in the field, and speculates about future directions for research. This book includes contributions by experts from different but related areas of research including constraint programming, decision theory, operations research, SAT, artificial intelligence, as well as others. These diverse perspectives are actively combined and contrasted in order to evaluate their relative advantages. This volume presents techniques for hybrid modeling, integrated solving strategies including global constraints, decomposition techniques, use of relaxations, and search strategies including tree search local search and metaheuristics. Various applications of the techniques presented as well as supplementary computational tools are also discussed.

This book constitutes the thoroughly refereed post-conference proceedings of the 9th International Conference on Learning and Optimization, LION 9, which was held in Lille, France, in January 2015. The 31 contributions presented were carefully reviewed and selected for inclusion in this book. The papers address all fields between machine learning, artificial intelligence, mathematical programming and algorithms for hard optimization problems. Special focus is given to algorithm selection and configuration, learning, fitness landscape, applications, dynamic optimization, multi-objective, max-clique problems, bayesian optimization and global optimization, data mining and - in a special session - also on dynamic optimization.