The Exploration Exploitation Dilemma

This paper reviews the extant literature on the exploration/exploitation dilemma. Based on a systematic analysis of structural, behavioural, systemic and temporal solutions, the authors are able to show that the learning literature continues to struggle with the question of how exactly an organization can separate exploration and exploitation and at the same time enable necessary knowledge exchange and cooperation between these two notions. Paying closer attention to networks might enable future research to answer this question. In particular, a combination of structural aspects of networks and social ties has the potential to explain how the solutions currently on offer can be implemented successfully, how organizations can combine several of them, and how they can shift between them.

This book constitutes the proceedings of the International Conference on Adaptive and Intelligent Systems, ICAIS 2011, held in Klagenfurt, Austria, in September 2011. The 36 full papers included in these proceedings together with the abstracts of 4 invited talks, were carefully reviewed and selected from 72 submissions. The contributions are organized under the following topical sections: incremental learning; adaptive system architecture; intelligent system engineering; data mining and pattern recognition; intelligent agents; and computational intelligence.

This paper argues that the dilemma of exploration versus exploitation also exits in case of new ventures (start-ups) especially in their growth phase, and thus raises the need for investigating the mechanisms of how ambidexterity is managed in the growth phase of new ventures. For this we discuss the challenges of managing the duality of exploration and exploitation, and how this is relevant in a new venture's growth context. The paper highlights how the top management characteristics and behavior may influence the balancing of this duality, and how it may affect the firm's performance. The paper also suggests potential research areas on the issue discussed.

The Palgrave Encyclopedia of Strategic Management has been written by an international team of leading academics, practitioners and rising stars and contains almost 550 individually commissioned entries. It is the first resource of its kind to pull together such a comprehensive overview of the field and covers both the theoretical and more empirically/practitioner oriented side of the discipline.

This handbook presents state-of-the-art research in reinforcement learning, focusing on its applications in the control and game theory of dynamic systems and future directions for related research and technology. The contributions gathered in this book deal with challenges faced when using learning and adaptation methods to solve academic and industrial problems, such as optimization in dynamic environments with single and multiple agents, convergence and performance analysis, and online implementation. They explore means by which these difficulties can be solved, and cover a wide range of related topics including: deep learning; artificial intelligence; applications of game theory; mixed modality learning; and multi-agent reinforcement learning. Practicing engineers and scholars in the field of machine learning, game theory, and autonomous control will find the Handbook of Reinforcement Learning and Control to be thought-provoking, instructive and informative.

The 33rd Annual German Conference on Arti?cial Intelligence (KI 2010) took place at the Karlsruhe Institute of Technology KIT, September 21–24, 2010, under the motto “Anthropomatic Systems.” In this volume you will ?nd the keynote paper and 49 papers of oral and poster presentations. The papers were selected from 73 submissions, resulting in an acceptance rate of 67%. As usual at the KI conferences, two entire days were allocated for targeted workshops—seventhis year—andone tutorial. The workshopand tutorialma- rials are not contained in this volume, but the conference website, www.ki2010.kit.edu,will provide information and references to their contents. Recent trends in AI research have been focusing on anthropomatic systems, which address synergies between humans and intelligent machines. This trend is emphasized through the topics of the overall conference program. They include learning systems, cognition, robotics, perception and action, knowledge rep- sentation and reasoning, and planning and decision making. Many topics deal with uncertainty in various scenarios and incompleteness of knowledge. Summarizing, KI 2010 provides a cross section of recent research in modern AI methods and anthropomatic system applications. We are very grateful that Jos ́ edel Mill ́ an, Hans-Hellmut Nagel, Carl Edward Rasmussen, and David Vernon accepted our invitation to give a talk.

The Handbook of the Psychology of Aging, Seventh Edition, provides a basic reference source on the behavioral processes of aging for researchers, graduate students, and professionals. It also provides perspectives on the behavioral science of aging for researchers and professionals from other disciplines. The book is organized into four parts. Part 1 reviews key methodological and analytical issues in aging research. It examines some of the major historical influences that might provide explanatory mechanisms for a better understanding of cohort and period differences in psychological aging processes. Part 2 includes chapters that discuss the basics and nuances of executive function; the history of the morphometric research on normal brain aging; and the neural changes that occur in the brain with aging. Part 3 deals with the social and health aspects of aging. It covers the beliefs that individuals have about how much they can control various outcomes in their life; the impact of stress on health and aging; and the interrelationships between health disparities, social class, and aging. Part 4 discusses the emotional aspects of aging; family caregiving; and mental disorders and legal capacities in older adults. Contains all the main areas of psychological gerontological research in one volume Entire section on neuroscience and aging Begins with a section on theory and methods Edited by one of the father of gerontology (Schaie) and contributors represent top scholars in gerontology

'Algorithms to Live By' looks at the simple, precise algorithms that computers use to solve the complex 'human' problems that we face, and discovers what they can tell us about the nature and origin of the mind.

In combination with Deep Neural Networks (DNNs), several Reinforcement Learning (RL) algorithms such as "Q-learning" of "Policy Gradient" are now able to achieve super-human performaces on most Atari Games as well as the game of Go. Despite these outstanding and promising achievements, such Deep Reinforcement Learning (DRL) algorithms require millions of samples to perform well, thus limiting their deployment to all applications where data acquisition is costly. The lack of sample efficiency of DRL can partly be attributed to the use of DNNs, which are known to be data-intensive in the training phase. But more importantly, it can be attributed to the type of Reinforcement Learning algorithm used, which only perform a very inefficient undirected exploration of the environment. For instance, Q-learning and Policy Gradient rely on randomization for exploration. In most cases, this strategy turns out to be very ineffective to properly balance the exploration needed to discover unknown and potentially highly rewarding regions of the environment, with the exploitation of rewarding regions already identified as such. Other RL approaches with theoretical guarantees on the exploration-exploitation trade-off have been investigated. It is sometimes possible to formally prove that the performances almost match the theoretical optimum. This line of research is inspired by the Multi-Armed Bandit literature, with many algorithms relying on the same underlying principle often referred as "optimism in the face of uncertainty". Even if a significant effort has been made towards understanding the exploration-exploitation dilemma generally, many questions still remain open. In this thesis, we generalize existing work on exploration-exploitation to different contexts with different amounts of prior knowledge on the learning problem. We introduce several algorithmic improvements to current state-of-the-art approaches and derive a new theoretical analysis which allows us to answer several open questions of the literature. We then relax the (very common although not very realistic) assumption that a path between any two distinct regions of the environment should always exist. Relaxing this assumption highlights the impact of prior knowledge on the intrinsic limitations of the exploration-exploitation dilemma. Finally, we show how some prior knowledge such as the range of the value function or a set of macro-actions can be efficiently exploited to speed-up learning. In this thesis, we always strive to take the algorithmic complexity of the proposed algorithms into account. Although all these algorithms are somehow computationally "efficient", they all require a planning phase and therefore suffer from the well-known "curse of dimensionality" which limits their applicability to real-world problems. Nevertheless, the main focus of this work is to derive general principles that may be combined with more heuristic approaches to help overcome current DRL flaws.

The significantly expanded and updated new edition of a widely used text on reinforcement learning, one of the most active research areas in artificial intelligence. Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives while interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the field's key ideas and algorithms. This second edition has been significantly expanded and updated, presenting new topics and updating coverage of other topics. Like the first edition, this second edition focuses on core online learning algorithms, with the more mathematical material set off in shaded boxes. Part I covers as much of reinforcement learning as possible without going beyond the tabular case for which exact solutions can be found. Many algorithms presented in this part are new to the second edition, including UCB, Expected Sarsa, and Double Learning. Part II extends these ideas to function approximation, with new sections on such topics as artificial neural networks and the Fourier basis, and offers expanded treatment of off-policy learning and policy-gradient methods. Part III has new chapters on reinforcement learning's relationships to psychology and neuroscience, as well as an updated case-studies chapter including AlphaGo and AlphaGo Zero, Atari game playing, and IBM Watson's wagering strategy. The final chapter discusses the future societal impacts of reinforcement learning.