Cover not available
Part of
Why Gesture?: How the hands function in speaking, thinking and communicating
Edited by Ruth Breckinridge Church, Martha W. Alibali and Spencer D. Kelly
[Gesture Studies 7] 2017
► pp. 39–58

Chapter 3
Function and processing of gesture in the context of language

| Radboud University Nijmegen | Max Planck Institute for Psycholinguistics

Most research focuses function of gesture independent of its link to the speech it accompanies and the coexpressive functions it has together with speech. This chapter instead approaches gesture in relation to its communicative function in relation to speech, and demonstrates how it is shaped by the linguistic encoding of a speaker’s message. Drawing on crosslinguistic research with adults and children as well as bilinguals on iconic/pointing gesture production it shows that the specific language speakers use modulates the rate and the shape of the iconic gesture production of the same events. The findings challenge the claims aiming to understand gesture’s function for “thinking only” in adults and during development.

Article outline
  • General introduction
  • Cross-linguistic evidence for the effect of spoken language on gesture
  • Experimental evidence showing effects of language on gesture
  • Linguistic effects on gesture during development across languages
  • General conclusions
  • Notes
  • References
Published online: 26 April 2017
DOI logo
https://doi.org/10.1075/gs.7.03ozy
References
Allen, S., Özyürek, A., Kita, S., Brown, A., Furman, R., Ishizuka, T., & Fujii, M.
2007 “Language-specific and universal influences in children’s packaging of manner and path: A comparison of English, Japanese, and Turkish.” Cognition 102 (1): 16–48. DOI logoGoogle Scholar
Bates, E.
1976 Language and Context: The Acquisition of Pragmatics. New York, NY: Academic Press.Google Scholar
Bates, E., Benigni, L., Bretherton, I., Camaioni, L., & Volterra, V.
1979 The Emergence of Symbols: Cognition-Communication in Infancy. New York, NY: Academic PressGoogle Scholar
Bavelas, Janet B., & Nicole Chovil
2000 “Visible acts of meaning. An integrated message model of language use in face-to-face dialogue.” Journal of Language and Social Psychology 19: 163–194. DOI logoGoogle Scholar
Beilock, S. L. & Goldin-Meadow, S.
2010 “Gesture changes thought by grounding it in action.” Psychological Science 21: 1605–1610. DOI logoGoogle Scholar
Brown, P.
2008 “Verb specificity and argument realization in Tzeltal child language.” In Crosslinguistic Perspectives on Argument Structure, M. Bowerman & P. Brown (eds), 167–189. New York, NY: Lawrence Erlbaum.Google Scholar
Campisi, E., & Özyürek, A.
2013 “Iconicity as a communicative strategy: Recipient design in multimodal demonstrations for adults and children.” Journal of Pragmatics 47: 14–27. DOI logoGoogle Scholar
Cartmill, E. A., Demir, Ö. E., Goldin-Meadow, S.
2012 “Studying gesture.” In Research Methods in Child Language: A Practical Guide, E. Hoff (ed), Oxford: Wiley-Blackwell. DOI logoGoogle Scholar
Choi, S., & Bowerman, M.
1991 “Learning to express motion events in English and Korean: The influence of language-specific lexicalization patterns.” Cognition 41: 83–121. DOI logoGoogle Scholar
Chu, M. and Kita, S.
2016 “Co-thought and co-speech gestures are generated from the same action generation process.” Journal of Experimental Psychology, Learning, Memory and Cognitio. 42(2): 257–270.Google Scholar
Clark, H.
1996 Using Language. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Colletta, J., Guidetti, M., Capirci, O., Cristilli, C., Demir, O., Kunene-Nicolas, R., Levine, S.
2014 “Effects of age and language on co-speech gesture production: an investigation of French, American, and Italian children's narratives.” Journal of Child Language: 1–21.Google Scholar
Cook, S. W., & Tanenhaus, M. K.
2009 “Embodied communication: Speakers’ gestures affect listeners’ actions.” Cognition 113: 98–104. DOI logoGoogle Scholar
Defina, R.
2016 “Do serial verb constructions describe single events? A study of co-speech gestures in Avatime.” Language 92(4): 890–910. DOI logoGoogle Scholar
Demir, Ö. E., So, W.C., Özyürek A., & Goldin-Meadow, S.
2012 “Turkish- and English-speaking children display sensitivity to perceptual context in the referring expressions they produce in speech and gesture.” Language and Cognitive Processes 27 (6): 844–867. DOI logoGoogle Scholar
Enfield, N.J.
2009 The Anatomy of Meaning: Speech, Gesture, and Composite Utterances. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Floyd, S.
2016 “Modally hybrid grammar? Celestial pointing for time-of-day reference in Nheengatú.” Language 92(1): 31–64. DOI logoGoogle Scholar
Furman, R., Kuntay, A. & Özyürek, A.
2013 “Early language-specificity of children's event encoding in speech and gesture: Evidence from caused motion in Turkish.” Language, Cognition and Neuroscience 29: 620–634. DOI logoGoogle Scholar
Gentner, D.
1982 “Why are nouns learned before verbs: Linguistic relativity versus natural partitioning.” In Language Development: Language, Thought and Culture Vol. 2, S.A. Kuczaj (ed), 301–334. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
Goldin-Meadow, S., McNeill, D., & Singleton, J.
1996 “Silence is liberating: Removing the handcuffs on grammatical expression in the manual modality.” Psychological Review 103: 34–55. DOI logoGoogle Scholar
Goldin-Meadow, S., So, C., Ozyurek, A. & Mylander, C.
2008 “The natural order of events: How speakers of different languages represent events nonverbally.” Proceedings of the National Academy of Sciences 105(27): 9163–9168. DOI logoGoogle Scholar
Göksun, T., Hirsh-Pasek, K., & Golinkoff, R. M.
2010 “How do preschoolers express cause in gesture and speech?” Cognitive Development 25(1): 56–68. DOI logoGoogle Scholar
Greenfield, P., & Smith, J.
1976 The Structure of Communication in Early Language Development. New York, NY: Academic Press.Google Scholar
Gu, Y. Mol, L., Hoetjes, M. & Swerts, M.
2014 “Does language shape the production and perception of gestures? A study on late Chinese-English bilinguals’ conceptions about time.” In Proceedings of the 36th Annual Meeting of the Cognitive Science Society P. Bello, M. Guarini, M. McShane, & B. Scassellati (eds),Google Scholar
Gullberg, M., Hendricks, H. & Hickmann, M.
2008 “Learning to talk and gesture about motion in French.” First Language 28(2): 200–236. DOI logoGoogle Scholar
Gullberg, M.
2011 “Language-specific encoding of placement events in gestures.” In Event Representation in Language and Cognition, J. Bohnemeyer & E. Pederson (eds), 166–188. Cambridge, MA: Cambridge University Press.Google Scholar
Gullberg, M., & Narasimhan, B.
2010 “What gestures reveal about how semantic distinctions develop in Dutch children’s placement verbs.” Cognitive Linguistics 21(2): 239–262. DOI logoGoogle Scholar
Hickmann, M., Taranne, P. & Bonnet, P.
2009 “Motion in first language acquisition: Manner and path in French and English child language.” Journal of Child Language 36(4): 705–741. DOI logoGoogle Scholar
Holler, J., & Beattie, G.
2003 “How iconic gestures and speech interact in the representation of meaning: are both aspects really integral to the process?” Semiotica 146: 81–116.Google Scholar
Hostetter, A. B., & Alibali, M. W.
2008 “Visible embodiment: Gestures as simulated action.” Psychonomic Bulletin & Review 15: 495–514. DOI logoGoogle Scholar
Iverson, J. M., Capirci, O., & Caselli, M. C.
1994 “From communication to language in two modalities.” Cognitive Development 9(1): 23–43. DOI logoGoogle Scholar
Iverson, J. M., Capirci, O., Volterra, V., & Goldin-Meadow, S.
2008 “Learning to talk in a gesture-rich world: Early communication in Italian vs. American children.” First Language 28(2): 164–181. DOI logoGoogle Scholar
Iverson, J. M., & Goldin-Meadow, S.
2005 “Gesture paves the way for language development.” Psychological Science 16(5): 367–371. DOI logoGoogle Scholar
Kelly, S. D., Özyürek, A., & Maris, E.
2010 “Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension.” Psychological Science 21: 260–267.Google Scholar
Kemmerer, D.
2014 The Cognitive Neuroscience of Language. Psychology Press.Google Scholar
Kendon, A.
2004 Gesture: Visible Action as Utterance. Cambridge, MA: Cambridge University Press. DOI logoGoogle Scholar
Ketrez, N., & Aksu-Koc, A.
2002 “Acquisition of noun and verb categories in Turkish.” In Studies in TurkishLlinguistics: Proceedings of the Xth International Conference on Turkish Linguistics, A. S. Ozsoy, D. Akar, M. Nakipoglu-Demiralp, E. Erguvanlı-Taylan, & A. Koc (eds), 239–246. Istanbul: Bogazic i University Press.Google Scholar
Kita, S., & Özyürek, A.
2003 “What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking.” Journal of Memory and Language 48(1): 16–32. DOI logoGoogle Scholar
Kita, S., Özyürek, A., Allen, S., Brown, A., Furman, R., & Ishizuka, T.
2007 “Relations between syntactic encoding and co-speech gestures: Implications for a model of speech and gesture production.” Language and Cognitive Processes 22(8): 1212–1236. DOI logoGoogle Scholar
Krahmer, E. and Swerts, M.
2007 “The effects of visual beats on prosodic prominence: Acoustic analyses, auditory perception and visual perception.” Journal of Memory and Language 57: 396–414. DOI logoGoogle Scholar
Krauss, R. M., Dushay, R. A., Chen, Y., & Rauscher, F.
1995 “The communicative value of conversational hand gestures.” Journal of Experimental Social Psychology 31: 533–552. DOI logoGoogle Scholar
Küntay, A., & Slobin, D. I.
1996 “Listening to a Turkish mother: Some puzzles for acquisition.” In Social Interaction, Social Context and Language, D. I. Slobin, J. Gerhardt, A. Kyratzis, & J. Guo (eds), 265–286. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Levelt, P.
1989 Speaking. MIT Press. Cambridge.Google Scholar
Liszkowski, U., Brown, P., Callaghan, T., Takada, A., & De Vos, C.
2012 “A prelinguistic gestural universal of human communication.” Cognitive Science 36(4): 698–713. DOI logoGoogle Scholar
McNeill, David
1985 “So you think gestures are nonverbal?” Psychological Review 92(3): 350–371. DOI logoGoogle Scholar
McNeill, D.
1992 Hand and Mind. Chicago, IL: University of Chicago Press.Google Scholar
Melinger, Alissa & Willem J. M. Levelt
2004 “Gesture and the communicative intention of the speaker.” Gesture 4(2): 119–141. DOI logoGoogle Scholar
Namy, L. L., Campbell, A. L., & Tomasello, M.
2004 “The changing role of iconicity in nonverbal symbol learning: A U-Shaped trajectory in the acquisition of arbitrary gestures.” Journal of Cognition and Development 5(1): 37–57. DOI logoGoogle Scholar
Nicoladis, E.
2002 “Some gestures develop in conjunction with spoken language development and others don’t: Evidence from bilingual preschoolers.” Journal of Nonverbal Behavior 26(4): 241–266. DOI logoGoogle Scholar
Nicoladis, E., Mayberry, R. I., & Genesee, F.
1999 “Gesture and early bilingual development.” Developmental Psychology 35(2): 514–526. DOI logoGoogle Scholar
Özçalışkan, Ş. & Slobin, D. I.
1999 “Learning ‘how to search for the frog’: Expression of manner of motion in English, Spanish and Turkish.” In Proceedings of the 23rd Boston University Conference on Language Development, A. Greenhill, H. Littlefield & C. Tano (eds), 541–552. Somerville, MA: Cascadilla Press.Google Scholar
Özçalışkan, Ş, & Goldin-Meadow, S.
2005 “Gesture is at the cutting edge of early language development.” Cognition 96(3): B101–B113. DOI logoGoogle Scholar
Özçalışkan, Ş.
2007 “Metaphors we move by: Children’s developing understanding of metaphorical motion in typologically distinct languages.” Metaphor & Symbol 22(2): 147–168. DOI logoGoogle Scholar
2009 “Learning to talk about spatial motion in language-specific ways.” In Cross-Linguistic Approaches to the Psychology of Language: Research in the Tradition of DanIsaac Slobin, J. Guo, E. Lieven, S. Ervin-Tripp, N. Budwig, K. Nakamura & Ş. Özçalışkan (eds), 263–276. New York NY: Psychology Press.Google Scholar
Özçalışkan, Ş, & Goldin-Meadow, S.
2009 “When gesture-speech combinations do and do not index linguistic change.” Language and Cognitive Processes, 24(2): 190–217. DOI logoGoogle Scholar
Özçalışkan, Ş. & Goldin-Meadow, S.
2011 “Is there an iconic gesture spurt at 26 months?” In Integrating Gestures: The Interdisciplinary Nature of Gesture, G. Stam & M. Ishino (eds), (website pub) Amsterdam: John Benjamins. DOI logoGoogle Scholar
Özçalışkan, Ş.
2012 “When gesture does and does not follow speech in describing motion.” In Online Supplement for the Proceedings of the 36th Boston University Conference on Language Development, A. Biller, E. Chung, & A. Kimball (eds).Google Scholar
Özçalışkan, Ş., Gentner, D., & Goldin-Meadow, S.
2013 “Do iconic gestures pave the way for children’s early verbs?” Applied Psycholinguistics 35: 1143–1162. DOI logoGoogle Scholar
Özçalışkan, Ş., Lucero, C., & Goldin-Meadow, S.
2015 “Is seeing gesture necessary to gesture like a native speaker?” Psychological Science, Google Scholar
Özyürek, A., Kita, S., Allen, S., Furman, R., & Brown, A.
2005 “How does linguistic framing of events influence co-speech gestures? Insights from crosslinguistic variations and similarities.” Gesture 5(1/2): 219–240. DOI logoGoogle Scholar
Özyürek, A., Kita, S., Allen, S., Furman, R., Brown, A., & Ishizuka, T.
2008 “Development of cross-linguistic variation in speech and gesture: Motion events in English and Turkish.” Developmental Psychology 44(4): 1040–1054. DOI logoGoogle Scholar
Özyürek, A., Furman, R., & Goldin-Meadow, S.
2014 “On the way to language: Event segmentation in homesign and gesture.” Journal of Child Language 29: 620–634.Google Scholar
Özyürek, A.
in press). “Crosslinguistic variation in children’s multimodal utterances.” In Sources of Variation in Language Acquisition, M. Kail, M. Hickman, & E. Veneziano eds TILAR Series Amsterdam Benjamins
Papafragou, A., Massey, C., & Gleitman, L.
2002 “Shake, rattle, ‘n’ roll: The representation of motion in language and cognition.” Cognition 84: 189–219. DOI logoGoogle Scholar
Peeters, D., Chu, M., Holler, J., Hagoort, P., & Özyürek, A.
2015 “Electrophysiological and kinematic correlates of communicative intent in the planning and production of pointing gestures and speech.” Journal of Cognitive Neuroscience 27(12): 2352–2368. DOI logoGoogle Scholar
Rowe, M., & Goldin-Meadow, S.G.
2009 “Differences in early gesture explain SES disparities in child vocabulary size at school entry.” Science 323: 951–953. DOI logoGoogle Scholar
Schubotz, L., Holler, J., & Özyürek, A.
2015 “Age-related differences in multi-modal audience design: Young, but not old speakers, adapt speech and gestures to their addressee's knowledge.” In Proceedings of the 4th GESPIN – Gesture & Speech in Interaction Conference, G. Ferré, & M. Tutton (eds), Nantes: Université of Nantes.Google Scholar
Schulman, Bari Wieselman
2004 “A Crosslinguistic Investigation of the Speech-gesture Relationship in Motion Event Descriptions.” PhD diss., University of Chicago, Department of Psychology.Google Scholar
Talmy, L.
1985 “Lexicalization patterns: Semantic structure in lexical forms.” In Language Typology and Semantic Description. Vol. III: Grammatical Categories and the Lexicon, T. Shopen (ed), 36–149. Cambridge: Cambridge University Press.Google Scholar
Wesp, R., Hesse, J., Keutmann, D., & Wheaton, K.
2001 “Gestures maintain spatial imagery.” The American Journal of Psychology 114: 591–600. DOI logoGoogle Scholar
Cited by

Cited by 7 other publications

Azar, Zeynep, Ad Backus & Aslı Özyürek
2020. Language contact does not drive gesture transfer: Heritage speakers maintain language specific gesture patterns in each language. Bilingualism: Language and Cognition 23:2 ► pp. 414 ff. DOI logo
Billot-Vasquez, Kiana, Zhongwen Lian, Yukari Hirata & Spencer D. Kelly
2020. Emblem Gestures Improve Perception and Evaluation of Non-native Speech. Frontiers in Psychology 11 DOI logo
Graziano, Maria & Marianne Gullberg
2024. Providing evidence for a well-worn stereotype: Italians and Swedes do gesture differently. Frontiers in Communication 9 DOI logo
Hostetter, Autumn B. & Martha W. Alibali
2019. Gesture as simulated action: Revisiting the framework. Psychonomic Bulletin & Review 26:3 ► pp. 721 ff. DOI logo
Sandler, Wendy
2018. The Body as Evidence for the Nature of Language. Frontiers in Psychology 9 DOI logo
Schubotz, Louise, Aslı Özyürek & Judith Holler
2019. Age-related differences in multimodal recipient design: younger, but not older adults, adapt speech and co-speech gestures to common ground. Language, Cognition and Neuroscience 34:2 ► pp. 254 ff. DOI logo
Ünal, Ercenur, Ezgi Mamus & Aslı Özyürek
2023. Multimodal encoding of motion events in speech, gesture and cognition. Language and Cognition ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 12 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.