Article published In:
GestureVol. 4:2 (2005) ► pp.119–141
Gesture and the communicative intention of the speaker
This paper aims to determine whether iconic tracing gestures produced while speaking constitute part of the speaker’s communicative intention. We used a picture description task in which speakers must communicate the spatial and color information of each picture to an interlocutor. By establishing the necessary minimal content of an intended message, we determined whether speech produced with concurrent gestures is less explicit than speech without gestures. We argue that a gesture must be communicatively intended if it expresses necessary information that was nevertheless omitted from speech. We found that speakers who produced iconic gestures representing spatial relations omitted more required spatial information from their descriptions than speakers who did not gesture. These results provide evidence that speakers intend these gestures to communicate. The results have implications for the cognitive architectures that underlie the production of gesture and speech.
Keywords: gesture, interaction, speech production
Cited by
Cited by 86 other publications
Alviar, Camila, Rick Dale & Alexia Galati
2019. Complex Communication Dynamics: Exploring the Structure of an Academic Talk. Cognitive Science 43:3 
Amada, Joichiro, Yuki Okafuji, Takahiro Wada, Jun Baba, Junya Nakanishi & Yuichiro Yoshikawa
2021. 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), ► pp. 1313 ff.
Asalıoğlu, Esma Nur & Tilbe Göksun
2023. The role of hand gestures in emotion communication: Do type and size of gestures matter?. Psychological Research 87:6 ► pp. 1880 ff.
Auer, Peter & Angelika Bauer
2011. Multimodality in aphasic conversation. Journal of Interactional Research in Communication Disorders 2:2 ► pp. 215 ff.
Bavelas, Janet, Jennifer Gerwing, Chantelle Sutton & Danielle Prevost
2008. Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language 58:2 ► pp. 495 ff.
Bonvillian, John D., Nicole Kissane Lee, Tracy T. Dooley & Filip T. Loncke
2020. Simplified Signs,
Boulenger, Véronique, Livio Finos, Eric Koun, Roméo Salemme, Clément Desoche & Alice C. Roy
2022. Up right, not right up: Primacy of verticality in both language and movement. Frontiers in Human Neuroscience 16
Brabanter, Philippe De
2010. Uttering Sentences Made Up of Words and Gestures. In Explicit Communication, ► pp. 199 ff.
Brunyé, Tad T., David N. Rapp & Holly A. Taylor
2008. Representational flexibility and specificity following spatial descriptions of real-world environments. Cognition 108:2 ► pp. 418 ff.
Chieffi, Sergio, Claudio Secchi & Maurizio Gentilucci
2009. Deictic word and gesture production: Their interaction. Behavioural Brain Research 203:2 ► pp. 200 ff.
Chui, Kawai, Chia-Ying Lee, Kanyu Yeh & Pei-Chun Chao
2018. Semantic processing of self-adaptors, emblems, and iconic gestures: An ERP study. Journal of Neurolinguistics 47 ► pp. 105 ff.
Clough, Sharice & Melissa C. Duff
2020. The Role of Gesture in Communication and Cognition: Implications for Understanding and Treating Neurogenic Communication Disorders. Frontiers in Human Neuroscience 14
Clough, Sharice, Victoria-Grace Padilla, Sarah Brown-Schmidt & Melissa C. Duff
2023. Intact speech-gesture integration in narrative recall by adults with moderate-severe traumatic brain injury. Neuropsychologia 189 ► pp. 108665 ff.
Cohen, Cathy & Ciara R. Wigham
2019. A comparative study of lexical word search in an audioconferencing and a videoconferencing condition. Computer Assisted Language Learning 32:4 ► pp. 448 ff.
Cravotta, Alice, Pilar Prieto & M. Grazia Busà
2021. Exploring the effects of restraining the use of gestures on narrative speech. Speech Communication 135 ► pp. 25 ff.
Cruz, Marisa, Marc Swerts & Sónia Frota
2017. The role of intonation and visual cues in the perception of sentence types: Evidence from European Portuguese varieties. Laboratory Phonology: Journal of the Association for Laboratory Phonology 8:1 ► pp. 23 ff.
de Beer, Carola, Marcella Carragher, Karin van Nispen, Katharina Hogrefe, Jan P. de Ruiter & Miranda L. Rose
2017. How Much Information Do People With Aphasia Convey via Gesture?. American Journal of Speech-Language Pathology 26:2 ► pp. 483 ff.
de Ruiter, Jan P., Adrian Bangerter & Paula Dings
2012. The Interplay Between Gesture and Speech in the Production of Referring Expressions: Investigating the Tradeoff Hypothesis. Topics in Cognitive Science 4:2 ► pp. 232 ff.
de Ruiter, Jan Peter
2006. Can gesticulation help aphasic people speak, or rather, communicate?. Advances in Speech Language Pathology 8:2 ► pp. 124 ff.
Di Pastena, Angela, Loris Tamara Schiaratura & Françoise Askevis-Leherpeux
2015. Joindre le geste à la parole : les liens entre la parole et les gestes co-verbaux. L’Année psychologique Vol. 115:3 ► pp. 463 ff.
Di Pastena, Angela, Loris Tamara Schiaratura & Françoise Askevis-Leherpeux
2015. Joindre le geste à la parole : les liens entre la parole et les gestes co-verbaux. L’Année psychologique 115:03 ► pp. 463 ff.
Di Pastena, Angela, Loris Tamara Schiaratura & Françoise Askevis-Leherpeux
2015. Joindre le geste à la parole : les liens entre la parole et les gestes co-verbaux. L’Année psychologique 115:03 ► pp. 463 ff.
Emir Özder, Levent, Demet Özer & Tilbe Göksun
2023. Gesture use in L1-Turkish and L2-English: Evidence from emotional narrative retellings. Quarterly Journal of Experimental Psychology 76:8 ► pp. 1797 ff.
Esteve-Gibert, Núria & Pilar Prieto
2014. Infants temporally coordinate gesture-speech combinations before they produce their first words. Speech Communication 57 ► pp. 301 ff.
Ferstl, Ylva, Michael Neff & Rachel McDonnell
2019. Motion, Interaction and Games, ► pp. 1 ff.
Ferstl, Ylva, Michael Neff & Rachel McDonnell
2020. Adversarial gesture generation with realistic gesture phasing. Computers & Graphics 89 ► pp. 117 ff.
Feyereisen, Pierre
2006. How could gesture facilitate lexical access?. Advances in Speech Language Pathology 8:2 ► pp. 128 ff.
Fu, Wai-Tat, Laura D'Andrea & Sven Bertel
2013. Effects of Communication Methods on Communication Patterns and Performance in a Remote Spatial Orientation Task. Spatial Cognition & Computation 13:2 ► pp. 150 ff.
Ghorbani, Saeed, Ylva Ferstl, Daniel Holden, Nikolaus F. Troje & Marc‐André Carbonneau
2023. ZeroEGGS: Zero‐shot Example‐based Gesture Generation from Speech. Computer Graphics Forum 42:1 ► pp. 206 ff.
Goldin-Meadow, Susan & Martha Wagner Alibali
2013. Gesture's Role in Speaking, Learning, and Creating Language. Annual Review of Psychology 64:1 ► pp. 257 ff.
Gullberg, Marianne
2006. Handling Discourse: Gestures, Reference Tracking, and Communication Strategies in Early L2. Language Learning 56:1 ► pp. 155 ff.
Gullberg, Marianne
2010. Methodological reflections on gesture analysis in second language acquisition and bilingualism research. Second Language Research 26:1 ► pp. 75 ff.
Gullberg, Marianne & Sotaro Kita
2009. Attention to Speech-Accompanying Gestures: Eye Movements and Information Uptake. Journal of Nonverbal Behavior 33:4 ► pp. 251 ff.
Hoetjes, Marieke, Ruud Koolen, Martijn Goudbeek, Emiel Krahmer & Marc Swerts
2015. Reduction in gesture during the production of repeated references. Journal of Memory and Language 79-80 ► pp. 1 ff.
Hogrefe, Katharina, Wolfram Ziegler, Susanne Wiesmayer, Nicole Weidinger & Georg Goldenberg
2013. The actual and potential use of gestures for communication in aphasia. Aphasiology 27:9 ► pp. 1070 ff.
Holler, Judith, Linda Drijvers, Afrooz Rafiee & Asifa Majid
2022. Embodied Space‐pitch Associations are Shaped by Language. Cognitive Science 46:2
Holler, Judith & Katie Wilkin
2011. Co-Speech Gesture Mimicry in the Process of Collaborative Referring During Face-to-Face Dialogue. Journal of Nonverbal Behavior 35:2 ► pp. 133 ff.
Hostetter, Autumn B.
2014. Action Attenuates the Effect of Visibility on Gesture Rates. Cognitive Science 38:7 ► pp. 1468 ff.
Hostetter, Autumn B. & Martha W. Alibali
2019. Gesture as simulated action: Revisiting the framework. Psychonomic Bulletin & Review 26:3 ► pp. 721 ff.
Hughes-Berheim, Sarah S., Laura M. Morett & Raymond Bulger
2020. Semantic Relationships Between Representational Gestures and Their Lexical Affiliates Are Evaluated Similarly for Speech and Text. Frontiers in Psychology 11
INAZAWA, Keisuke & Akihiro KASHIHARA
2022. Designing and Evaluating Presentation Avatar for Promoting Self-Review. IEICE Transactions on Information and Systems E105.D:9 ► pp. 1546 ff.
Ishino, Tatsuya, Mitsuhiro Goto & Akihiro Kashihara
2018. Proceedings of the 6th International Conference on Human-Agent Interaction, ► pp. 67 ff.
Ishino, Tatsuya, Mitsuhiro Goto & Akihiro Kashihara
2022. Robot lecture for enhancing presentation in lecture. Research and Practice in Technology Enhanced Learning 17:1
Jarmolowicz-Nowikow, Ewa
2012. Are Pointing Gestures Induced by Communicative Intention?. In Cognitive Behavioural Systems [Lecture Notes in Computer Science, 7403], ► pp. 377 ff.
Kashihara, Akihiro, Tatsuya Ishino & Mitsuhiro Goto
2019. Robot Lecture for Enhancing Non-verbal Behavior in Lecture. In Artificial Intelligence in Education [Lecture Notes in Computer Science, 11626], ► pp. 128 ff.
Kelly, Spencer D., Sarah Ward, Peter Creigh & James Bartolotti
2007. An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language 101:3 ► pp. 222 ff.
Kim, Suyeon & Sookyung Cho
2017. How a Tutor Uses Gesture for Scaffolding: A Case Study on L2 Tutee's Writing. Discourse Processes 54:2 ► pp. 105 ff.
Kimbara, Irene
2008. Gesture Form Convergence in Joint Description. Journal of Nonverbal Behavior 32:2 ► pp. 123 ff.
Kistner, Judith, Lucy T. Dipper & Jane Marshall
2019. The use and function of gestures in word-finding difficulties in aphasia. Aphasiology 33:11 ► pp. 1372 ff.
Kopp, Stefan & Kirsten Bergmann
2012. Individualized Gesture Production in Embodied Conversational Agents. In Human-Computer Interaction: The Agency Perspective [Studies in Computational Intelligence, 396], ► pp. 287 ff.
KOPP, STEFAN, KIRSTEN BERGMANN & IPKE WACHSMUTH
2008. MULTIMODAL COMMUNICATION FROM MULTIMODAL THINKING — TOWARDS AN INTEGRATED MODEL OF SPEECH AND GESTURE PRODUCTION. International Journal of Semantic Computing 02:01 ► pp. 115 ff.
Leonteva, Anna V., Alan Cienki & Olga V. Agafonova
2023. Metaphoric gestures in simultaneous interpreting. Russian Journal of Linguistics 27:4 ► pp. 820 ff.
Marshall, Jane, Wendy Best, Naomi Cocks, Madeline Cruice, Tim Pring, Gemma Bulcock, Gemma Creek, Nancy Eales, Alice Lockhart Mummery, Niina Matthews & Anna Caute
2012. Gesture and Naming Therapy for People With Severe Aphasia: A Group Study. Journal of Speech, Language, and Hearing Research 55:3 ► pp. 726 ff.
Mlakar, Izidor, Matej Rojc, Simona Majhenič & Darinka Verdonik
Mlakar, Izidor, Darinka Verdonik, Simona Majhenič & Matej Rojc
2023. Understanding conversational interaction in multiparty conversations: the EVA Corpus. Language Resources and Evaluation 57:2 ► pp. 641 ff.
Mol, Lisette, Emiel Krahmer & Mieke van de Sandt-Koenderman
2013. Gesturing by Speakers With Aphasia: How Does It Compare?. Journal of Speech, Language, and Hearing Research 56:4 ► pp. 1224 ff.
Nathan, Mitchell J., Amelia Yeo, Rebecca Boncoddo, Autumn B. Hostetter & Martha W. Alibali
Park, Hee Sun & Xiaowen Guan
2009. Cross-Cultural Comparison of Verbal and Nonverbal Strategies of Apologizing. Journal of International and Intercultural Communication 2:1 ► pp. 66 ff.
Peeters, David, Mingyuan Chu, Judith Holler, Peter Hagoort & Aslı Özyürek
2015. Electrophysiological and Kinematic Correlates of Communicative Intent in the Planning and Production of Pointing Gestures and Speech. Journal of Cognitive Neuroscience 27:12 ► pp. 2352 ff.
Pritchard, Madeleine, Lucy Dipper, Gary Morgan & Naomi Cocks
2015. Language and iconic gesture use in procedural discourse by speakers with aphasia. Aphasiology 29:7 ► pp. 826 ff.
Rose, Miranda L.
2013. Releasing the Constraints on Aphasia Therapy: The Positive Impact of Gesture and Multimodality Treatments. American Journal of Speech-Language Pathology 22:2
Rossi-Arnaud, Clelia, Pietro Spataro & Emiddia Longobardi
2012. Effects of pointing on the recall of simultaneous and sequential visuospatial arrays: a role for retrieval strategies?. Psychological Research 76:6 ► pp. 699 ff.
Saha, Sriparna & Amit Konar
2018. A Study on Static Hand Gesture Recognition Using Type-1 Fuzzy Membership Function. In Applied Computational Intelligence and Soft Computing in Engineering [Advances in Systems Analysis, Software Engineering, and High Performance Computing, ], ► pp. 105 ff.
Scharp, Victoria L., Connie A. Tompkins & Jana M. Iverson
2007. Gesture and aphasia: Helping hands?. Aphasiology 21:6-8 ► pp. 717 ff.
Sekine, Kazuki, Miranda L. Rose, Abby M. Foster, Michelle C. Attard & Lucette E. Lanyon
2013. Gesture production patterns in aphasic discourse: In-depth description and preliminary predictions. Aphasiology 27:9 ► pp. 1031 ff.
Shattuck-Hufnagel, Stefanie
2019. Toward an (even) more comprehensive model of speech production planning. Language, Cognition and Neuroscience 34:9 ► pp. 1202 ff.
Smith, Lindsey W. & Roberto A. Delgado
2013. Considering the role of social dynamics and positional behavior in gestural communication research. American Journal of Primatology 75:9 ► pp. 891 ff.
Smith, Lindsey W. & Roberto A. Delgado
2015. Body language: The interplay between positional behavior and gestural signaling in the genus Pan and its implications for language evolution. American Journal of Physical Anthropology 157:4 ► pp. 592 ff.
Stefanatos, Gerry A.
2012. Developmental neuropsychology of autism. In The Autism Spectrum, ► pp. 59 ff.
Stefanatos, Gerry A. & Ida Sue Baron
2011. The Ontogenesis of Language Impairment in Autism: A Neuropsychological Perspective. Neuropsychology Review 21:3 ► pp. 252 ff.
Stevens, James & Yang Zhang
2014. Brain mechanisms for processing co-speech gesture: A cross-language study of spatial demonstratives. Journal of Neurolinguistics 30 ► pp. 27 ff.
Tzeng, Christina Y., Laura L. Namy & Lynne C. Nygaard
2019. Communicative Context Affects Use of Referential Prosody. Cognitive Science 43:11
TÖRET, Zahide & Selda ÖZDEMİR
2021. Jest Kullanımının Görme Yetersizliğinden Etkilenmiş ve Gören Yetişkinlerin Leksikal Ulaşım Sürecine Etkisinin Karşılaştırılması. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Özel Eğitim Dergisi 22:2 ► pp. 395 ff.
Vajrabhaya, Prakaiwan & Eric Pederson
Wei, Carolyn
2006. 2006 IEEE International Professional Communication Conference, ► pp. 299 ff.
Willems, Roel M., Aslı Özyürek & Peter Hagoort
2007. When Language Meets Action: The Neural Integration of Gesture and Speech. Cerebral Cortex 17:10 ► pp. 2322 ff.
Wilson, Amy Alexandra, Michael D. Boatright & Melanie Landon-Hays
2014. Middle School Teachers’ Discipline-Specific Use of Gestures and Implications for Disciplinary Literacy Instruction. Journal of Literacy Research 46:2 ► pp. 234 ff.
Yang, Zhaojun & Shrikanth S. Narayanan
2017. Modeling Dynamics of Expressive Body Gestures In Dyadic Interactions. IEEE Transactions on Affective Computing 8:3 ► pp. 369 ff.
Zhao, Zhong, Haiming Tang, Xiaobin Zhang, Zhipeng Zhu, Jiayi Xing, Wenzhou Li, Da Tao, Xingda Qu & Jianping Lu
2023. Characteristics of Visual Fixation in Chinese Children with Autism During Face-to-Face Conversations. Journal of Autism and Developmental Disorders 53:2 ► pp. 746 ff.
Zhao, Zhong, Jiayi Xing, Xiaobin Zhang, Xingda Qu, Xinyao Hu & Jianping Lu
2022. Random and Short-Term Excessive Eye Movement in Children with Autism During Face-to-Face Conversation. Journal of Autism and Developmental Disorders 52:8 ► pp. 3699 ff.
Özer, Demet & Tilbe Göksun
2020. Gesture Use and Processing: A Review on Individual Differences in Cognitive Resources. Frontiers in Psychology 11
Özer, Demet, Tilbe Göksun & Anjan Chatterjee
2019. Differential roles of gestures on spatial language in neurotypical elderly adults and individuals with focal brain injury. Cognitive Neuropsychology 36:5-6 ► pp. 282 ff.
Özyürek, Aslı
2017. Chapter 3. Function and processing of gesture in the context of language. In Why Gesture? [Gesture Studies, 7], ► pp. 39 ff.
Özyürek, Aslı, Roel M. Willems, Sotaro Kita & Peter Hagoort
2007. On-line Integration of Semantic Information from Speech and Gesture: Insights from Event-related Brain Potentials. Journal of Cognitive Neuroscience 19:4 ► pp. 605 ff.
Ünal, Ercenur, Francie Manhardt & Aslı Özyürek
2022. Speaking and gesturing guide event perception during message conceptualization: Evidence from eye movements. Cognition 225 ► pp. 105127 ff.
Żygis, Marzena & Susanne Fuchs
2023. Communicative constraints affect oro-facial gestures and acoustics: Whispered vs normal speech. The Journal of the Acoustical Society of America 153:1 ► pp. 613 ff.
This list is based on CrossRef data as of 30 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.