Conceptual Motorics Generation And Evaluation Of Communicative Robot Gesture

How do humans perceive communicative gesture behavior in robots? Although gesture is a crucial feature of social interaction, this research question is still largely unexplored in the field of social robotics. The present work thus sets out to investigate how robot gesture can be used to design and realize more natural and human-like communication capabilities for social robots. The adopted approach is twofold. Firstly, the technical challenges encountered when implementing a speech-gesture generation model on a robotic platform are addressed. The realized framework enables a humanoid robot to produce finely synchronized speech and co-verbal hand and arm gestures. In contrast to many existing systems, these gestures are not limited to a predefined repertoire of motor actions but are flexibly generated at run-time. Secondly, the achieved expressiveness is exploited in controlled experiments to gain a deeper understanding of how robot gesture might impact human experience and evaluation of human-robot interaction. The findings reveal that participants evaluate the robot more positively when non-verbal behaviors such as hand and arm gestures are displayed along with speech. Surprisingly, this effect was particularly pronounced when the robot's gesturing behavior was partly incongruent with speech. These findings contribute new insights into human perception of communicative robot gesture and ultimately support the presented approach of endowing social robots with such non-verbal behaviors.

This book constitutes the refereed proceedings of the 7th International Conference on Social Robotics, ICSR 2015, held in Paris, France, in October 2015. The 70 revised full papers presented were carefully reviewed and selected from 126 submissions. The papers focus on the interaction between humans and robots and the integration of robots into our society and present innovative ideas and concepts, new discoveries and improvements, novel applications on the latest fundamental advances in the core technologies that form the backbone of social robotics, distinguished developmental projects, as well as seminal works in aesthetic design, ethics and philosophy, studies on social impact and influence pertaining to social robotics, and its interaction and communication with human beings and its social impact on our society.

This book constitutes the refereed proceedings of the 5th International Conference on Social Robotics, ICSR 2013, held in Bristol, UK, in October 2013. The 55 revised full papers and 13 abstracts were carefully reviewed and selected from 108 submissions and are presented together with one invited paper. The papers cover topics such as human-robot interaction, child development and care for the elderly, as well as technical issues underlying social robotics: visual attention and processing, motor control and learning.

This monograph presents analyses of filled and unfilled pauses, cut-offs, repair, discourse markers and other phenomena often referred to as disfluencies in the context of advanced language learners' PowerPoint presentations. It adopts a multimodal perspective to demonstrate the functions of these elements in interaction. Paired with gaze shifts, pointing gestures and posture shifts, they act as facilitators of joint visual orientation, mutual understanding, and accountable actions. Therefore, this volume suggests the name cofluency to reflect their potential functionality. Cofluencies are essential elements of multimodal chunks and multimodal patterns, and these are building blocks of a multimodal turn-taking mechanism for presentations. These concepts are illustrated and discussed based on excerpts from naturally occurring classroom data.

This book constitutes the refereed proceedings of the Third International Conference on Social Robotics, ICSR 2011, held in Amsterdam, The Netherlands, in November 2011. The 23 revised full papers were carefully selected during two rounds of reviewing and improvement from 51 submissions. The papers are organized in topical sections on social interaction with robots; nonverbal interaction with social robots; robots in society; social robots in education; affective interaction with social robots; robots in the home.

Having a natural interaction makes a significant difference in a successful human-robot interaction (HRI). The natural HRI refers to both human multimodal behavior understanding and robot verbal or non-verbal behavior generation. Humans can naturally communicate through spoken dialogue and non-verbal behaviors. Hence, a robot should perceive and understand human behaviors so as to be capable of producing a natural multimodal and spontaneous behavior that matches the social context. In this thesis, we explore human behavior understanding and robot behavior generation for natural HRI. This includes multimodal human emotion recognition with visual information extracted from RGB-D and thermal cameras and non-verbal multimodal robot behavior synthesis.Emotion recognition based on multimodal human behaviors during HRI can help robots understand user states and exhibit a natural social interaction. In this thesis, we explored multimodal emotion recognition with thermal facial information and 3D gait data in HRI scene when the emotion cues from thermal face and gait data are difficult to disguise. A multimodal database with thermal face images and 3D gait data was built through the HRI experiments. We tested the various unimodal emotion classifiers (i.e., CNN, HMM, Random Forest model, SVM) and one decision-based hybrid emotion classifier on the database for offline emotion recognition. We also explored an online emotion recognition system with limited capability in the real-time HRI setting. Interaction plays a critical role in skills learning for natural communication. Robots can get feedback during the interaction to improve their social abilities in HRI.To improve our online emotion recognition system, we developed an interactive robot learning (IRL) model with the human in the loop. The IRL model can apply the human verbal feedback to label or relabel the data for retraining the emotion recognition model in a long-term interaction situation. After using the interactive robot learning model, the robot could obtain a better emotion recognition accuracy in real-time HRI.The human non-verbal behaviors such as gestures and face action occur spontaneously with speech, which leads to a natural and expressive interaction. Speech-driven gesture and face action generation are vital to enable a social robot to exhibit social cues and conduct a successful HRI. This thesis proposes a new temporal GAN (Generative Adversarial Network) architecture for a one-to-many mapping from acoustic speech representation to the humanoid robot's corresponding gestures. We also developed an audio-visual database to train the speaking gesture generation model. The database includes the speech audio data extracted directly from the videos and the associated 3D human pose data extracted from 2D RGB images. The generated gestures from the trained co-speech gesture synthesizer can be applied to social robots with arms. The evaluation result shows the effectiveness of our generative model for speech-driven robot gesture generation. Moreover, we developed an effective speech-driven facial action synthesizer based on GAN, i.e., given an acoustic speech, a synchronous and realistic 3D facial action sequence is generated. A mapping between the 3D human facial actions to real robot facial actions that regulate the Zeno robot facial expression is completed. The application of co-speech non-verbal robot behaviors (gesture and face action) synthesis for the social robot can make a friendly and natural human-robot interaction.

Computational Human-Robot Interaction provides the reader with a systematic overview of the field of Human-Robot Interaction over the past decade, with a focus on the computational frameworks, algorithms, techniques, and models currently used to enable robots to interact with humans.

This book covers an approach to conversational informatics which encompasses science and technology for understanding and augmenting conversation in the network age. A major challenge in engineering is to develop a technology for conveying not just messages but also underlying wisdom. Relevant theories and practices in cognitive linguistics and communication science, as well as techniques developed in computational linguistics and artificial intelligence, are discussed.