Dynamic Sound Rendering Of Complex Environments

During the past twenty years many studies have been conducted in the field of auralization, which aimsat rendering audible the results of an acoustic simulation. These studies have mainly focused on thepropagation algorithms and the sound field audio rendering for complex environments. Currently, muchresearch concentrates on real-time audio rendering.This thesis addresses the problematic of real-time audio rendering of complex environments accordingto four axes: sound propagation, Digital Signal Processing (DSP), spatial perception of sound andcomputational optimizations. In the field of propagation, a method that aims at analyzing the varietyof existing algorithms is proposed. This method yields two algorithms dedicated to the real-time propagationof both specular and diffuse information. In the field of DSP, the auralization is performed withan efficient binaural spatialization module for the most significant specular information, and a GPUconvolution algorithm for the diffuse sound field auralization. The most significant paths are extractedthanks to a perceptive model based on temporal and spatial masking of the specular contributions.Finally, the implementation of these algorithms on recent computer architectures, taking advantage ofthe parallel processing of the new CPUs, and the benefits of GPUs for DSP calculations is presented.

Analogous to visualization, the auralization of virtual environments describes the simulation of sound propagation inside enclosures where methods of Geometrical Acoustics are mostly applied for a high-quality synthesis of aural stimuli that go along with a certain realistic behavior. In the course of this thesis, the design and implementation of the real-time room acoustics simulation software RAVEN will be described, which is a vital part of the implemented 3D sound-rendering system of RWTH Aachen University's immersive Virtual Reality system. RAVEN relies on present-day knowledge of room acoustical simulation techniques and enables a physically accurate auralization of sound propagation in complex environments including important wave effects such as sound scattering, airborne sound insulation between rooms and sound diffraction. Despite this realistic sound field rendering, not only spatially distributed and freely movable sound sources and receivers are supported at runtime but also modifications and manipulations of the environment itself. All major features are evaluated by investigating both the overall accuracy of the room acoustics simulation and the performance of implemented algorithms, and possibilities for further simulation optimizations are identified by assessing empirical studies of subjects operating in immersive environments

Musical Performance covers many aspects like Musical Acoustics, Music Psychology, or motor and prosodic actions. It deals with basic concepts of the origin or music and its evolution, ranges over neurocognitive foundations, and covers computational, technological, or simulation solutions. This volume gives an overview about current research in the foundation of musical performance studies on all these levels. Recent concepts of synchronized systems, evolutionary concepts, basic understanding of performance as Gestalt patterns, theories of chill as performance goals or historical aspects are covered. The neurocognitive basis of motor action in terms of music, musical syntax, as well as therapeutic aspects are discussed. State-of-the-art applications in performance realizations, like virtual room acoustics, virtual musicians, new concepts of real-time physical modeling using complex performance data as input or sensor and gesture studies with soft- and hardware solutions are presented. So although the field is still much larger, this volume presents current trends in terms of understanding, implementing, and perceiving performance.

Immersive Sound: The Art and Science of Binaural and Multi-Channel Audio provides a comprehensive guide to multi-channel sound. With contributions from leading recording engineers, researchers, and industry experts, Immersive Sound includes an in-depth description of the physics and psychoacoustics of spatial audio as well as practical applications. Chapters include the history of 3D sound, binaural reproduction over headphones and loudspeakers, stereo, surround sound, height channels, object-based audio, soundfield (ambisonics), wavefield synthesis, and multi-channel mixing techniques. Knowledge of the development, theory, and practice of spatial and multi-channel sound is essential to those advancing the research and applications in the rapidly evolving fields of 3D sound recording, augmented and virtual reality, gaming, film sound, music production, and post-production.

Understanding Virtual Reality: Interface, Application, and Design, Second Edition, arrives at a time when the technologies behind virtual reality have advanced dramatically in their development and deployment, providing meaningful and productive virtual reality applications. The aim of this book is to help users take advantage of ways they can identify and prepare for the applications of VR in their field, whatever it may be. The included information counters both exaggerated claims for VR, citing dozens of real-world examples. By approaching VR as a communications medium, the authors have created a resource that will remain relevant even as the underlying technologies evolve. You get a history of VR, along with a good look at systems currently in use. However, the focus remains squarely on the application of VR and the many issues that arise in application design and implementation, including hardware requirements, system integration, interaction techniques and usability. Features substantive, illuminating coverage designed for technical or business readers and the classroom Examines VR's constituent technologies, drawn from visualization, representation, graphics, human-computer interaction and other fields Provides (via a companion website) additional case studies, tutorials, instructional materials and a link to an open-source VR programming system Includes updated perception material and new sections on game engines, optical tracking, VR visual interface software and a new glossary with pictures

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments.

The advent of augmented reality technologies used to assist human operators in complex manipulative operations—has brought an urgency to research into the modeling and training of human skills in Virtual Environments. However, modeling a specific act still represents a challenge in cognitive science. The same applies for the control of humanoid robots and the replication of skilled behavior of avatars in Virtual Environments. Skill Training in Multimodal Virtual Environments presents the scientific background, research outcomes, engineering developments, and evaluation studies conducted during the five years (2006-2011) of the project SKILLS–Multimodal Interfaces for Capturing and Transfer of Skill, funded by the European Commission under its 6th Framework Programme for Research and Technological Development. The SKILLS project evaluated how to exploit robotics and virtual environment technologies for the training of specific skills. This book details the novel approach used in the study to cope with skill acquisition, setting aside the mainstream assumptions of common computer-assisted training simulators. It explores how the SKILLS approach generated new training scenarios that allow users to practice new experiences in the performance of the devised task. Using a carefully designed approach that balances science with practicality, the book explores how virtual and augmented reality systems can be designed to address the skill transfer and training in different application contexts. The application of the same roadmap to skills originating from domains such as sports, rehabilitation, industrial environment, and surgery sets this book apart. It demonstrates how technology-oriented training conditions can yield better results than more traditional training conditions.

We are delighted to welcome readers to the proceedings of the 6th Pacific-Rim Conference on Multimedia (PCM). The first PCM was held in Sydney, Australia, in 2000. Since then, it has been hosted successfully by Beijing, China, in 2001, Hsinchu, Taiwan, in 2002, Singapore in 2003, and Tokyo, Japan, in 2004, and finally Jeju, one of the most beautiful and fantastic islands in Korea. This year, we accepted 181 papers out of 570 submissions including regular and special session papers. The acceptance rate of 32% indicates our commitment to ensuring a very high-quality conference. This would not be possible without the full support of the excellent Technical Committee and anonymous reviewers that provided timely and insightful reviews. We would therefore like to thank the Program Committee and all reviewers. The program of this year reflects the current interests of the PCM’s. The accepted papers cover a range of topics, including, all aspects of multimedia, both technical and artistic perspectives and both theoretical and practical issues. The PCM 2005 program covers tutorial sessions and plenary lectures as well as regular presentations in three tracks of oral sessions and a poster session in a single track. We have tried to expand the scope of PCM to the artistic papers which need not to be strictly technical.

This book gives a broad overview of research on sound simulation driven by a variety of applications. Vibrating objects produce sound, which then propagates through a medium such as air or water before finally being heard by a listener. As a crucial sensory channel, sound plays a vital role in many applications. There is a well-established research community in acoustics that has studied the problems related to sound simulation for six decades. Some of the earliest work was motivated by the design of concert halls, theaters, or lecture rooms with good acoustic characteristics. These problems also have been investigated in other applications, including noise control and sound design for urban planning, building construction, and automotive applications. Moreover, plausible or realistic sound effects can improve the sense of presence in a virtual environment or a game. In these applications, sound can provide important clues such as source directionality and spatial size. The book first surveys various sound synthesis methods, including harmonic synthesis, texture synthesis, spectral analysis, and physics-based synthesis. Next, it provides an overview of sound propagation techniques, including wave-based methods, geometric-based methods, and hybrid methods. The book also summarizes various techniques for sound rendering. Finally, it surveys some recent trends, including the use of machine learning methods to accelerate sound simulation and the use of sound simulation techniques for other applications such as speech recognition, source localization, and computer-aided design.