Talk to the Virtual Hands: Self-Animated Avatars Improve Communication in Head-Mounted Display Virtual Environments
PLoS ONEPublished 12 October 2011Open access
Trevor J. Dodds, Betty J. Mohler, HH Bülthoff
Citations85
SJR quartileQ1
SJR score0.80
SNIP1.07
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The studies show how virtual reality can be used to systematically study the influence of body gestures and it is important that nonverbal communication is bidirectional (real nonverbal feedback in addition to non verbal communication from the describing participant).
Abstract
Taken together, the studies show how (a) virtual reality can be used to systematically study the influence of body gestures; (b) it is important that nonverbal communication is bidirectional (real nonverbal feedback in addition to nonverbal communication from the describing participant); and (c) there are differences in the amount of body gestures that participants use with and without the head-mounted display, and we discuss possible explanations for this and ideas for future investigation.
Keywords
PsychologyComputer Science
Modern Language JournalLearning Strategies in Second Language Acquisition
2,939 Citations1992Carol Hosenfeld, J. Michael O’Malley +1 more
Cambridge University Press eBooksLearning Strategies in Second Language Acquisition
2,693 Citations1990J. Michael O’Malley, Anna Uhl Chamot
The series editors' preface explains how cognitive theory applies to second language acquisition and some of the strategies used by second language learners.
Human Communication ResearchThe Proteus Effect: The Effect of Transformed Self-Representation on Behavior
2,016 Citations2007Nick Yee, Jeremy N. Bailenson
Contemporary Sociology A Journal of ReviewsConducting Interaction: Patterns of Behavior in Focused Encounters.
1,639 Citations1992Judee K. Burgoon, Adam Kendon
PRESENCE Virtual and Augmented RealityThe Demographics, Motivations, and Derived Experiences of Users of Massively Multi-User Online Graphical Environments
1,273 Citations2006Nick Yee
An exploratory factor analysis revealed a five factor model of user motivations—Achievement, Relationship, Immersion, Escapism, and Manipulation—illustrating the multifaceted appeal of these online environments.
PLoS ONEFirst Person Experience of Body Transfer in Virtual Reality
1,010 Citations2010Mel Slater, Bernhard Spanlang +2 more
It is shown that a first person perspective of a life-sized virtual human female body that appears to substitute the male subjects' own bodies was sufficient to generate a body transfer illusion, supporting the notion that bottom-up perceptual mechanisms can temporarily override top down knowledge resulting in a radical illusion of transfer of body ownership.
Communication ResearchThe Proteus Effect
772 Citations2009Nick Yee, Jeremy N. Bailenson +1 more
Together, these two studies show that their virtual bodies can change how the authors interact with others in actual avatar-based online communities as well as in subsequent face-to-face interactions.
Trends in Cognitive SciencesThe role of gesture in communication and thinking
683 Citations1999Susan Goldin‐Meadow, Susan Goldin-Meadow
In this review, gesture is examined when it stands on its own, substituting for speech and clearly serving a communicative function, and when produced along with speech, which suggests these are not mutually exclusive functions.
PLoS ONEA Virtual Reprise of the Stanley Milgram Obedience Experiments
610 Citations2006Mel Slater, Angus Antley +6 more
The results show that in spite of the fact that all participants knew for sure that neither the stranger nor the shocks were real, the participants who saw and heard her tended to respond to the situation at the subjective, behavioural and physiological levels as if it were real.
ScienceDifferences in Early Gesture Explain SES Disparities in Child Vocabulary Size at School Entry
557 Citations2009Meredith L. Rowe, Susan Goldin‐Meadow
It is found that children from high-SES families frequently used gesture to communicate at 14 months, a relation that was explained by parent gesture use (with speech controlled).
Frontiers in Human NeuroscienceTowards a digital body: The virtual arm illusion
549 Citations2008Mel Slater
It is demonstrated that a virtual limb can be made to feel part of your body if appropriate multisensory correlations are provided, which opens up the possibility that an entire virtual body could be felt as one's own in future virtual reality applications or online games.
Acta PsychologicaMovement coordination in social interaction: Some examples described
460 Citations1970Adam Kendon
It is found that the flow of movement in the listener may be rhythmically coordinated with the speech and movements of the speaker, and how the way in which individuals may be in synchrony with one another can vary is shown.
PLoS ONEVirtual Hand Illusion Induced by Visuomotor Correlations
432 Citations2010María V. Sánchez-Vives, Bernhard Spanlang +3 more
Synchrony between visual and proprioceptive information along with motor activity is able to induce an illusion of ownership over a virtual arm, which has implications regarding the brain mechanisms underlying body ownership as well as the use of virtual bodies in therapies and rehabilitation.
NatureWhy people gesture when they speak
422 Citations1998Jana M. Iverson, Susan Goldin‐Meadow
It is shown that congenitally blind speakers gesture despite their lack of a visual model, even when they speak to a blind listener, because gestures require neither a model nor an observant partner.
Oxford University Press eBooksBeyond the Perception-Behavior Link: The Ubiquitous Utility and Motivational Moderators of Nonconscious Mimicry
265 Citations2006Tanya L. Chartrand, William W. Maddux +1 more
Mediated Interpersonal Communication
263 Citations2008Elly A. Konijn
A framework for classifying digital human representations and the role they play in Computer Mediated Communication is discussed and a theory called Transformed Social Interaction (TSI) is presented that explores how CMC allows people to interact in ways not possible face-toface.
The American Journal of PsychologyThe Role of Gestures in Spatial Working Memory and Speech
260 Citations2004Ezequiel Morsella, Robert M. Krauss
It was found participants gestured more when describing visual objects from memory and when describing objects that were difficult to remember and encode verbally, and gesture restriction produced dysfluent speech even when spatial memory was untaxed, suggesting that gestures can directly affect both spatial memory and lexical retrieval.
ACM Transactions on GraphicsSimulating virtual environments within virtual environments as the basis for a psychophysics of presence
222 Citations2010Mel Slater, Bernhard Spanlang +1 more
PRESENCE Virtual and Augmented RealityThe Effect of Viewing a Self-Avatar on Distance Judgments in an HMD-Based Virtual Environment
208 Citations2010Betty J. Mohler, Sarah H. Creem-Regehr +2 more
Using an action-based response measure, it is found that participants who explored near space while seeing a fully-articulated and tracked visual representation of themselves subsequently made more accurate judgments of absolute egocentric distance to locations ranging from 4 m to 6 m away from where they were standing than did participants who saw no avatar.
Consciousness and CognitionSpatial aspects of bodily self-consciousness
202 Citations2008Bigna Lenggenhager, Michaël Mouthon +1 more
It is shown that the unitary and localized character of the self can be experimentally separated from both the origin of the visual perspective and the location of the seen body, which is compatible with clinical data.
Routledge eBooksTheories and methods in mediated communication
169 Citations2003Steve Whittaker
Studying studies of mediated communication allow us to isolate the individual contribution of different nonverbal behaviours, such as gaze and gesture to communication, and help to clarify the overall influence of visual information in communications, and the effects of communication interactive processes such as feedback on communication.
EurographicsBuilding Expression into Virtual Characters
131 Citations2006Vinoba Vinayagamoorthy, Marco Gillies +5 more
Large linguistically-processed web corpora for multiple languages
123 Citations2006Marco Baroni, Adam Kilgarriff
This work provides Web access to the corpora in the query tool, the Sketch Engine, for German and Italian, with corpus sizes of over 1 billion words in each case.
The Journal of Visualization and Computer AnimationGaze and task performance in shared virtual environments
119 Citations2002Jeremy N. Bailenson, Andrew C. Beall +1 more
The data demonstrated that interactants in the rendered head movement condition rated a higher level of co-presence, liked each other more, looked at each other's heads more, and spoke for a lower percentage of time during the game, compared to the other two conditions.
PsycEXTRA DatasetAn experiment on fear of public speaking in virtual reality
116 Citations2001David-Paul Pertaub, Mel Slater +1 more
Results clearly show that not only is social anxiety induced by the audience, but the degree of anxiety experienced is directly related to the type of virtual audience feedback the speaker receives.
Can a virtual cat persuade you?
115 Citations2006Catherine Zanbaka, Paula Goolkasian +1 more
Cross gender interactions occurred across all conditions, suggesting that some of the gender stereotypes that occur with people may carry over to interaction with virtual characters in the design of persuasive human computer interfaces.
PRESENCE Virtual and Augmented RealityEffects of Handling Real Objects and Self-Avatar Fidelity on Cognitive Task Performance and Sense of Presence in Virtual Environments
111 Citations2003Benjamin Lok, Samir Naik +2 more
Man manipulating real objects in a VE brings task performance closer to that of real space, compared to manipulating virtual objects, and there was no signifi-cant difference in reported sense of presence, regardless of the self-avatar's visual fidelity or the presence of real objects.
Journal of Experimental Psychology Human Perception & PerformanceSpatiotemporal symmetry and multifractal structure of head movements during dyadic conversation.
97 Citations2009Kathleen T. Ashenfelter, Steven M. Boker +2 more
Windowed cross-correlation analyses found high peak correlation between conversants' head movements over short ( approximately 2-s) intervals and a high degree of nonstationarity, consistent with a view that symmetry is formed between conversant over short intervals and that this symmetry is broken at longer, irregular intervals.
Management International ReviewPracticing what We Preach
85 Citations2012Anne‐Wil Harzing, Isabel Metz
PRESENCE Virtual and Augmented RealityThe Usability of Collaborative Virtual Environments and Methods for the Analysis of Interaction
83 Citations2006Ralph Schroeder, Ilona Heldal +1 more
Two methods for analyzing interactions in collaborative virtual environments (CVEs) are described, one whereby quantitative data are captured, interaction is categorized into a number of activities, and statistical analysis can be performed on frequencies and sequences of events, and the other based on the transcription of individual fragments of interaction.
PRESENCE Virtual and Augmented RealityNew Method to Measure End-to-End Delay of Virtual Reality
81 Citations2010Massimiliano Di Luca
The results collected with oscillatory movements performed at different frequencies indicate that for some VR systems, the end-to-end delay might not be constant but could vary as a function of the oscillation frequency.
ACM Transactions on GraphicsSeeing is believing
63 Citations2010Cathy Ennis, Rachel McDonnell +1 more
Lie tracking
57 Citations2010William Steptoe, Anthony Steed +2 more
Two experiments investigating eye tracking in AMC demonstrate that observers of AMC can more accurately detect truth and deception when viewing avatars with added oculesic behavior driven by eye tracking, and uncovers systematic differences between truth telling and lying.
Beyond the Perception-Behavior Link: The Ubiquitous Utility and Motivational Moderators of Nonconscious Mimicry
50 Citations2004Tanya L. Chartrand, William W. Maddux +1 more
Simulating virtual environments within virtual environments as the basis for a psychophysics of presence
45 Citations2010Mel Slater, Bernhard Spanlang +1 more
This concept can be used as the basis for a psychophysics of presence in VEs, the sensation of being in the place depicted by the virtual environment displays (Place Illusion, PI), and also the illusion that events occurring in thevirtual environment are real (Plausibility Illusion, Psi).
Oxford University Press eBooksThe Glimpsed World: Unintended Communication and Unintended Perception
42 Citations2006Youjin Choi, Heather M. Gray +1 more
ACM Transactions on Applied PerceptionTalking bodies
26 Citations2009Rachel McDonnell, Cathy Ennis +2 more
Human sensitivity to the coordination and timing of conversational body language for virtual characters is investigated and detection performance was above chance for both conversation styles but more so for the polite conversations, where desynchronization was more noticeable.
The Glimpsed World: Unintended Communication and Unintended Perception
16 Citations2004Youjin Choi, Heather M. Gray +1 more
IEEE Computer Graphics and ApplicationsPracticing What We Preach: IEEE VR 2009 Virtual Program Committee Meeting
13 Citations2009Robert W. Lindeman, Dirk Reiners +1 more
The experience of conducting the program committee (PC) meeting for the IEEE VR 2009 conference in Second Life indicates generally positive feelings toward this meeting format, although several alterations will help improve interaction between attendees in future meetings.
Computers & GraphicsUsing mobile group dynamics and virtual time to improve teamwork in large-scale collaborative virtual environments
12 Citations2009Trevor J. Dodds, Roy A. Ruddle
The implementation of MGDs and VT and their evaluation using an urban planning task show how CVE interfaces can be improved for synchronous and asynchronous collaborations, and highlight possibilities for future research.