Performance Evaluation And Benchmarking Of Intelligent Systems

To design and develop capable, dependable, and affordable intelligent systems, their performance must be measurable. Scienti?c methodologies for standardization and benchmarking are crucial for quantitatively evaluating the performance of eme- ing robotic and intelligent systems’ technologies. There is currently no accepted standard for quantitatively measuring the performance of these systems against user-de?ned requirements; and furthermore, there is no consensus on what obj- tive evaluation procedures need to be followed to understand the performance of these systems. The lack of reproducible and repeatable test methods has precluded researchers working towards a common goal from exchanging and communic- ing results, inter-comparing system performance, and leveraging previous work that could otherwise avoid duplication and expedite technology transfer. Currently, this lack of cohesion in the community hinders progress in many domains, such as m- ufacturing, service, healthcare, and security. By providing the research community with access to standardized tools, reference data sets, and open source libraries of solutions, researchers and consumers will be able to evaluate the cost and be- ?ts associated with intelligent systems and associated technologies. In this vein, the edited book volume addresses performance evaluation and metrics for intel- gent systems, in general, while emphasizing the need and solutions for standardized methods. To the knowledge of the editors, there is not a single book on the market that is solely dedicated to the subject of performance evaluation and benchmarking of intelligent systems.

This book constitutes the thoroughly refereed post-conference proceedings of the 10th TPC Technology Conference on Performance Evaluation and Benchmarking, TPCTC 2018, held in conjunction with the 44th International Conference on Very Large Databases (VLDB 2018) in August 2018. The 10 papers presented were carefully reviewed and selected from numerous submissions. The TPC encourages researchers and industry experts to present and debate novel ideas and methodologies in performance evaluation, measurement, and characterization.

This book constitutes the thoroughly refereed post-conference proceedings of the 10th TPC Technology Conference on Performance Evaluation and Benchmarking, TPCTC 2018, held in conjunction with the 44th International Conference on Very Large Databases (VLDB 2018) in August 2018. The 10 papers presented were carefully reviewed and selected from numerous submissions. The TPC encourages researchers and industry experts to present and debate novel ideas and methodologies in performance evaluation, measurement, and characterization.

This book constitutes the refereed post-conference proceedings of the 12th TPC Technology Conference on Performance Evaluation and Benchmarking, TPCTC 2020, held in August 2020.The 8 papers presented were carefully reviewed and cover the following topics: testing ACID compliance in the LDBC social network benchmark; experimental performance evaluation of stream processing engines made easy; revisiting issues in benchmarking metric selection; performance evaluation for digital transformation; experimental comparison of relational and NoSQL document systems; a framework for supporting repetition and evaluation in the process of cloud-based DBMS performance benchmarking; benchmarking AI inference; a domain independent benchmark evolution model for the transaction processing performance council.

Are psychometric tests valid for a new reality of artificial intelligence systems, technology-enhanced humans, and hybrids yet to come? Are the Turing Test, the ubiquitous CAPTCHAs, and the various animal cognition tests the best alternatives? In this fascinating and provocative book, José Hernández-Orallo formulates major scientific questions, integrates the most significant research developments, and offers a vision of the universal evaluation of cognition. By replacing the dominant anthropocentric stance with a universal perspective where living organisms are considered as a special case, long-standing questions in the evaluation of behavior can be addressed in a wider landscape. Can we derive task difficulty intrinsically? Is a universal g factor - a common general component for all abilities - theoretically possible? Using algorithmic information theory as a foundation, the book elaborates on the evaluation of perceptual, developmental, social, verbal and collective features and critically analyzes what the future of intelligence might look like.

Many new technologies – like RFID, GPS, and sensor networks – that dominate innovative developments in logistics are based on the idea of autonomous cooperation and control. This self-organisational concept describes „...processes of decentralized decision-making in heterarchical structures. It presumes interacting elements in non-deterministic systems, which possess the capability and possibility to render decisions. The objective of autonomous cooperation and control is the achievement of increased robustness and positive emergence of the total system due to distributed and flexible coping with dynamics and complexity“ (Hülsmann & Windt, 2007). In order to underlie these technology-driven developments with a fundamental theoretical foundation this edited volume asks for contributions and limitations of applying the principles of autonomous cooperation and control to logistics processes and systems. It intends to identify, describe, and explain – in the context of production and distribution logistics – the effects on performance and robustness, the enablers and impediments for the feasibility, the essential cause-effect-relations, etc. of concepts, methods, technologies, and routines of autonomous cooperation and control in logistics. Therefore, the analyses collected in this edited volume aim to develop a framework for finding the optimal degree as well as the upper and lower boundaries of autonomous cooperation and control of logistics processes from the different perspectives of production technology, electronics and communication engineering, informatics and mathematics, as well as management sciences and economics.

The era of the fourth industrial revolution has fundamentally transformed the manufacturing landscape. Products are getting increasingly complex and customers expect a higher level of customization and quality. Manufacturing in the Era of 4th Industrial Revolution explores three technologies that are the building blocks of the next-generation advanced manufacturing.The first technology covered in Volume 1 is Additive Manufacturing (AM). AM has emerged as a very popular manufacturing process. The most common form of AM is referred to as 'three-dimensional (3D) printing'. Overall, the revolution of additive manufacturing has led to many opportunities in fabricating complex, customized, and novel products. As the number of printable materials increases and AM processes evolve, manufacturing capabilities for future engineering systems will expand rapidly, resulting in a completely new paradigm for solving a myriad of global problems.The second technology is industrial robots, which is covered in Volume 2 on Robotics. Traditionally, industrial robots have been used on mass production lines, where the same manufacturing operation is repeated many times. Recent advances in human-safe industrial robots present an opportunity for creating hybrid work cells, where humans and robots can collaborate in close physical proximities. This Cobots, or collaborative robots, has opened up to opportunity for humans and robots to work more closely together. Recent advances in artificial intelligence are striving to make industrial robots more agile, with the ability to adapt to changing environments and tasks. Additionally, recent advances in force and tactile sensing enable robots to be used in complex manufacturing tasks. These new capabilities are expanding the role of robotics in manufacturing operations and leading to significant growth in the industrial robotics area.The third technology covered in Volume 3 is augmented and virtual reality. Augmented and virtual reality (AR/VR) technologies are being leveraged by the manufacturing community to improve operations in a wide variety of ways. Traditional applications have included operator training and design visualization, with more recent applications including interactive design and manufacturing planning, human and robot interactions, ergonomic analysis, information and knowledge capture, and manufacturing simulation. The advent of low-cost solutions in these areas is accepted to accelerate the rate of adoption of these technologies in the manufacturing and related sectors.Consisting of chapters by leading experts in the world, Manufacturing in the Era of 4th Industrial Revolution provides a reference set for supporting graduate programs in the advanced manufacturing area.

Featuring a detailed analysis and presentation of innovative researches, methods, algorithms and technologies that deal with integrated intelligent systems for the efficient management of energy and indoor environment in buildings, this book encompasses the regulations, directives and standards regarding the energy and the indoor environment of buildings as well as a literature review and discussion on the current state-of-the-art for buildings’ energy efficiency classification. Maximizing reader insight into this topic with the aid of simulation models for buildings and energy audits at office buildings are presented including tables and figures with the detailed information regarding the parameters, inputs, outputs and the outcomes of the surveys. This book also outlines the development of a Virtual Building Dataset (VBD) of office buildings as an innovative benchmarking and classification tool. The proposed methodology overcomes the difficulties and time required for collecting the necessary massive building constructional and energy bills data by creating them virtually using efficient stochastic simulation and by taking into account all parameters that may affect the energy performance and indoor thermal comfort of office buildings. The knowledge and ideas conveyed by the book are supported with equations and algorithms and 137 colored figures and 55 tables, and features a rich bibliography, references and web sources. The book contains the basic knowledge undergraduate and especially postgraduate courses on the emergent subject of energy management and saving in buildings. The innovative aspects and guides of the book give serious opportunities to the postgraduate students in this scientific area to further develop their research skills and capabilities.