Article published In:
GestureVol. 2:1 (2002) ► pp.1–17
An experimental study of when and how speakers use gestures to communicate
This experiment expanded the visual availability paradigm by subsuming it under the broader principle of recipient design. We varied recipient design by asking speakers to describe a picture to someone who would see a videotape of their description or only hear an audiotape. Second, speakers described pictures that varied in verbal encodability. Finally, in addition to gestural rate, we analysed the redundancy of gestures with words. The results (N = 40) confirmed our predictions that speakers gesture at a higher rate and use a higher proportion of nonredundant gestures when their recipient would see their videotape; that they also use more nonredundant gestures when describing a picture for which they have a poor vocabulary; and that these two factors interact to produce the strongest effects when vocabulary is limited and the recipient would see the videotape. These effects support the hypothesis that speakers design their gestures to communicate to recipients.
Keywords: gestures, communication, speaker, encoding, recipient design
Cited by
Cited by 42 other publications
Abramov, Olga, Friederike Kern, Sofia Koutalidis, Ulrich Mertens, Katharina Rohlfing & Stefan Kopp
2021. The Relation Between Cognitive Abilities and the Distribution of Semantic Features Across Speech and Gesture in 4‐year‐olds. Cognitive Science 45:7 
Bavelas, Janet, Jennifer Gerwing & Sara Healing
2014. Effect of Dialogue on Demonstrations: Direct Quotations, Facial Portrayals, Hand Gestures, and Figurative References. Discourse Processes 51:8 ► pp. 619 ff.
Bavelas, Janet, Jennifer Gerwing, Chantelle Sutton & Danielle Prevost
2008. Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language 58:2 ► pp. 495 ff.
Bergmann, Kirsten
2013. Co-speech Gesture Generation for Embodied Agents and its Effects on User Evaluation. In Coverbal Synchrony in Human-Machine Interaction, ► pp. 223 ff.
Bergmann, Kirsten, Sebastian Kahl & Stefan Kopp
2013. Modeling the Semantic Coordination of Speech and Gesture under Cognitive and Linguistic Constraints. In Intelligent Virtual Agents [Lecture Notes in Computer Science, 8108], ► pp. 203 ff.
Bergmann, Kirsten & Stefan Kopp
2010. Systematicity and Idiosyncrasy in Iconic Gesture Use: Empirical Analysis and Computational Modeling. In Gesture in Embodied Communication and Human-Computer Interaction [Lecture Notes in Computer Science, 5934], ► pp. 182 ff.
Campisi, Emanuela & Asli Özyürek
2013. Iconicity as a communicative strategy: Recipient design in multimodal demonstrations for adults and children. Journal of Pragmatics 47:1 ► pp. 14 ff.
Drijvers, Linda & Judith Holler
2023. The multimodal facilitation effect in human communication. Psychonomic Bulletin & Review 30:2 ► pp. 792 ff.
Emir Özder, Levent, Demet Özer & Tilbe Göksun
2023. Gesture use in L1-Turkish and L2-English: Evidence from emotional narrative retellings. Quarterly Journal of Experimental Psychology 76:8 ► pp. 1797 ff.
Esposito, Anna, Jessica Vassallo, Antonietta M. Esposito & Nikolaos Bourbakis
2015. 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI), ► pp. 660 ff.
Gibbs,, Raymond W. & Teenie Matlock
2008. Metaphor, imagination, and simulation. In The Cambridge Handbook of Metaphor and Thought, ► pp. 161 ff.
Gullberg, Marianne
2010. Methodological reflections on gesture analysis in second language acquisition and bilingualism research. Second Language Research 26:1 ► pp. 75 ff.
Holler, Judith & Rachel Stevens
2007. The Effect of Common Ground on How Speakers Use Gesture and Speech to Represent Size Information. Journal of Language and Social Psychology 26:1 ► pp. 4 ff.
Holler, Judith, Kylie Turner & Trudy Varcianna
2013. It's on the tip of my fingers: Co-speech gestures during lexical retrieval in different social contexts. Language and Cognitive Processes 28:10 ► pp. 1509 ff.
Holler, Judith & Katie Wilkin
2009. Communicating common ground: How mutually shared knowledge influences speech and gesture in a narrative task. Language and Cognitive Processes 24:2 ► pp. 267 ff.
Holler, Judith & Katie Wilkin
2011. Co-Speech Gesture Mimicry in the Process of Collaborative Referring During Face-to-Face Dialogue. Journal of Nonverbal Behavior 35:2 ► pp. 133 ff.
Holler, Judith & Katie Wilkin
2011. An experimental investigation of how addressee feedback affects co-speech gestures accompanying speakers’ responses. Journal of Pragmatics 43:14 ► pp. 3522 ff.
Hostetter, Autumn B. & Catherine J. Skirving
2011. The Effect of Visual vs. Verbal Stimuli on Gesture Production. Journal of Nonverbal Behavior 35:3 ► pp. 205 ff.
Jarmolowicz-Nowikow, Ewa
2012. Are Pointing Gestures Induced by Communicative Intention?. In Cognitive Behavioural Systems [Lecture Notes in Computer Science, 7403], ► pp. 377 ff.
Jarmołowicz-Nowikow, Ewa
2015. How Poles indicate people and objects, and what they think of certain forms of pointing gestures. Lingua Posnaniensis 56:1 ► pp. 85 ff.
Karpiński, Maciej
2009. Preliminary Prosodic and Gestural Characteristics of Instructing Acts in Polish Task-Oriented Dialogues. In Cross-Modal Analysis of Speech, Gestures, Gaze and Facial Expressions [Lecture Notes in Computer Science, 5641], ► pp. 227 ff.
Kelly, Spencer, Kelly Byrne & Judith Holler
2011. Raising the Ante of Communication: Evidence for Enhanced Gesture Use in High Stakes Situations. Information 2:4 ► pp. 579 ff.
Kelly, Spencer D., Sarah Ward, Peter Creigh & James Bartolotti
2007. An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language 101:3 ► pp. 222 ff.
Kopp, Stefan & Kirsten Bergmann
2012. Individualized Gesture Production in Embodied Conversational Agents. In Human-Computer Interaction: The Agency Perspective [Studies in Computational Intelligence, 396], ► pp. 287 ff.
KOPP, STEFAN, KIRSTEN BERGMANN & IPKE WACHSMUTH
2008. MULTIMODAL COMMUNICATION FROM MULTIMODAL THINKING — TOWARDS AN INTEGRATED MODEL OF SPEECH AND GESTURE PRODUCTION. International Journal of Semantic Computing 02:01 ► pp. 115 ff.
Lavelle, M., P. G. T. Healey & R. McCabe
2013. Is Nonverbal Communication Disrupted in Interactions Involving Patients With Schizophrenia?. Schizophrenia Bulletin 39:5 ► pp. 1150 ff.
Majid, Asifa
2013. Making semantics and pragmatics “sensory”. Journal of Pragmatics 58 ► pp. 48 ff.
Melinger, Alissa & Sotaro Kita
2007. Conceptualisation load triggers gesture production. Language and Cognitive Processes 22:4 ► pp. 473 ff.
Minto-García, Aline, Elda A. Alva Canto & Natalia Arias-Trejo
2020. Mothers’ Use of Gestures and their Relationship to Children’s Lexical Production. Psychology of Language and Communication 24:1 ► pp. 175 ff.
O’Carroll, Sinead, Elena Nicoladis & Lisa Smithson
2015. The effect of extroversion on communication: Evidence from an interlocutor visibility manipulation. Speech Communication 69 ► pp. 1 ff.
Pfeiffer, Thies
2012. Interaction between Speech and Gesture: Strategies for Pointing to Distant Objects. In Gesture and Sign Language in Human-Computer Interaction and Embodied Communication [Lecture Notes in Computer Science, 7206], ► pp. 238 ff.
Rigoli, Lillian & Michael J. Spivey
2015. Real-Time Language Processing as Embodied and Embedded in Joint Action. In Attention and Vision in Language Processing, ► pp. 3 ff.
Rowbotham, Samantha, Judith Holler, Donna Lloyd & Alison Wearden
2012. How Do We Communicate About Pain? A Systematic Analysis of the Semantic Contribution of Co-speech Gestures in Pain-focused Conversations. Journal of Nonverbal Behavior 36:1 ► pp. 1 ff.
Rowbotham, Samantha, Judith Holler, Donna Lloyd & Alison Wearden
2014. Handling pain: The semantic interplay of speech and co-speech hand gestures in the description of pain sensations. Speech Communication 57 ► pp. 244 ff.
Rowbotham, Samantha, Donna M. Lloyd, Judith Holler & Alison Wearden
2015. Externalizing the Private Experience of Pain: A Role for Co-Speech Gestures in Pain Communication?. Health Communication 30:1 ► pp. 70 ff.
Rowbotham, Samantha J., Judith Holler, Alison Wearden & Donna M. Lloyd
2016. I see how you feel: Recipients obtain additional information from speakers’ gestures about pain. Patient Education and Counseling 99:8 ► pp. 1333 ff.
Schubotz, Louise, Aslı Özyürek & Judith Holler
2019. Age-related differences in multimodal recipient design: younger, but not older adults, adapt speech and co-speech gestures to common ground. Language, Cognition and Neuroscience 34:2 ► pp. 254 ff.
Sparrow, Karen, Christopher Lind & Willem van Steenbrugge
2020. Gesture, communication, and adult acquired hearing loss. Journal of Communication Disorders 87 ► pp. 106030 ff.
Streeck, Jürgen
2007. Homo Faber's Gestures. Journal of Linguistic Anthropology 17:1 ► pp. 130 ff.
Tetnowski, Jennifer Thompson, John A. Tetnowski & Jack S. Damico
2023. Looking at gesture: The reciprocal influence between gesture and conversation. Journal of Communication Disorders 106 ► pp. 106379 ff.
Wei, Carolyn
2006. 2006 IEEE International Professional Communication Conference, ► pp. 299 ff.
Yunus, Fajrian, Chloé Clavel & Catherine Pelachaud
2019. Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents, ► pp. 233 ff.
This list is based on CrossRef data as of 30 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.