Cover not available
Article published In:
Gesture
Vol. 21:2/3 (2022) ► pp.239–263

Co-speech gestures can interfere with learning foreign language words

ORCID logo | University of Alberta
| University of Alberta
| University of Alberta
Co-speech gestures can help the learning, processing, and memory of words and concepts, particularly motoric and spatial concepts such as verbs. The purpose of the present studies was to test whether co-speech gestures support the learning of words through gist traces of movement. We asked English monolinguals to learn 40 Cantonese words (20 verbs and 20 nouns). In two studies, we found support for the gist traces of congruent gestures being movement: participants who saw congruent gestures while hearing Cantonese words thought they had seen more verbs than participants in any other condition. However, gist traces were unrelated to the accurate recall of either nouns or verbs. In both studies, learning Cantonese words accompanied by congruent gestures tended to interfere with the learning of nouns (but not verbs). In Study 2, we ruled out the possibility that this interference was due either to gestures conveying representational information in another medium or to distraction from moving hands. We argue that gestures can interfere with learning foreign language words when they represent the referents (e.g., show shape or size) because learners must interpret the hands as something other than hands.
Keywords: congruent gestures, word learning, implicit memory, gist traces, representation
Article outline
  • Study 1
    • Method
      • Participants
      • Materials
      • Procedure
      • Scoring the recall task
    • Results
      • Discussion of Study 1
  • Study 2
    • Method
      • Participants
      • Materials and procedure
    • Results
      • Study 2a
      • Study 2b
      • Discussion of Study 2
    • General discussion
  • Acknowledgements
  • Note
  • References
Published online: 21 August 2023
DOI logo
https://doi.org/10.1075/gest.18020.nic
References (51)
References
Alibali, Martha. (2005). Gesture in spatial cognition: expressing, communicating, and thinking about spatial information. Spatial Cognition and Computation, 5 1, 307–331. DOI logoGoogle Scholar
Alibali, Martha W., Flevares, Lucia M., & Goldin-Meadow, Susan. (1997). Assessing knowledge conveyed in gesture: Do teachers have the upper hand? Journal of Educational Psychology, 89 1, 183–193. DOI logoGoogle Scholar
Andren, Mats & Blomberg, Johan. (2018). Children’s use of gesture and action with static and dynamic verbs. Language, Interaction, and Acquisition, 9 1, 22–39. DOI logoGoogle Scholar
Beattie, Geoffrey & Shovelton, Heather. (1999). Mapping the range of information contained in the iconic hand gestures that accompany spontaneous speech. Journal of Language and Social Psychology, 18 1, 438–462. DOI logoGoogle Scholar
Brainerd, Charles J. & Reyna, Valerie F. (2004). Fuzzy-trace theory and memory development. Developmental Review, 24 1, 396–439. DOI logoGoogle Scholar
Broaders, Sara C., Cook, Susan Wagner, Mitchell, Zachary, & Goldin-Meadow, Susan. (2007). Making children gesture brings out implicit knowledge and leads to learning. Journal of Experimental Psychology: General, 136 1, 539–550. DOI logoGoogle Scholar
Bub, Daniel N. & Masson, Michael E. J. (2006). Gestural knowledge evoked by objects as part of conceptual representations. Aphasiology, 20 1, 1112–1124. DOI logoGoogle Scholar
Cave, Carolyn Backer, Bost, Preston R., & Cobb, Ronald E. (1996). Effects of color and pattern on implicit and explicit picture memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22 1, 639–653.Google Scholar
Choi, Soojung & Lantolf, James P. (2008). Representation and embodiment of meaning in L2 communication: Motion events in speech and gesture of advanced L2 Korean and L2 English speakers. Studies in Second Language Acquisition, 30 1, 191–224. DOI logoGoogle Scholar
Church, R. Breckinridge & Goldin-Meadow, Susan. (1986). The mismatch between gesture and speech as an index of transitional knowledge. Cognition, 23 1, 43–71. DOI logoGoogle Scholar
de Nooijer, Jacqueline A., van Gog, Tamara, Paas, Fred, & Zwaan, Rolf A. (2014). Words in action: Using gestures to improve verb learning in primary school children. Gesture, 14 1, 46–69. DOI logoGoogle Scholar
Goldin-Meadow, Susan, Kim, San, & Singer, Melissa. (1999). What the teachers’ hands tell the students’ minds about math. Journal of Educational Psychology, 91 1, 720–730. DOI logoGoogle Scholar
Goldin-Meadow, Susan & Sandhofer, Catherine Momeni. (1999). Gesture conveys substantive information about a child’s thoughts to ordinary listeners. Developmental Science, 2 1, 67–74. DOI logoGoogle Scholar
Goldin-Meadow, Susan, So, Wing Chee, Özyürek, Aslı, & Mylander, Carolyn. (2008). The natural order of events: How speakers of different languages represent events nonverbally. Proceedings of the National Academy of Sciences, 105 1, 9163–9168. DOI logoGoogle Scholar
Goodrich, Whitney & Kam, Carla L. (2009). Co-speech gesture as input in verb learning. Developmental Science, 12 1, 81–87. DOI logoGoogle Scholar
Hadar, Uri & Krauss, Robert K. (1999). Iconic gestures: The grammatical categories of lexical affiliates. Journal of Neurolinguistics, 12 1, 1–12. DOI logoGoogle Scholar
Hostetter, Autumn B. & Alibali, Martha W. (2019). Gesture as simulated action: Revisiting the framework. Psychonomic Bulletin & Review, 26 1, 721–752. DOI logoGoogle Scholar
(2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin and Review, 15 1, 495–514. DOI logoGoogle Scholar
Howe, Mark L., Garner, Sarah R., Dewhurst, Stephen A., & Ball, Linden J. (2010). Can false memories prime problem solutions? Cognition, 117 1, 176–181. DOI logoGoogle Scholar
Kelly, Spencer D. & Church, R. Breckinridge. (1998). A comparison between children’s and adults’ ability to detect conceptual information conveyed through representational gestures. Child Development, 69 1, 85–93. DOI logoGoogle Scholar
Kelly, Spencer D., Kravitz, Corinne, & Hopkins, Michael. (2004). Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89 1, 253–260. DOI logoGoogle Scholar
Kelly, Spencer D., McDevitt, Tara, & Esch, Megan. (2009). Brief training with co-speech gesture lends a hand to word learning in a foreign language. Language and Cognitive Processing, 24 1, 313–334. DOI logoGoogle Scholar
Kendon, Adam. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), The relationship of verbal and non-verbal communication (pp. 207–228). The Hague, the Netherlands: Mouton. DOI logoGoogle Scholar
Kita, Sotaro & Özyürek, Aslı. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48 1, 16–32. DOI logoGoogle Scholar
Lazaraton, Anne. (2004). Gesture and speech in the vocabulary explanations of one ESL teacher: A microanalytic inquiry. Language Learning, 54 1, 79–117. DOI logoGoogle Scholar
Macedonia, Manuela, Müller, Karsten, & Friederici, Angela D. (2010a). Neural correlates of high performance in foreign language vocabulary learning. Mind, Brain, and Education, 4 1, 125–134. DOI logoGoogle Scholar
(2010b). The impact of iconic gestures on foreign language word learning and its neural substrate. Human Brain Mapping, 32 1, 982–998. DOI logoGoogle Scholar
Masson, Michael E. J., Bub, Daniel N., Newton-Taylor, Meaghan. (2008). Language-based access to gestural components of conceptual knowledge. The Quarterly Journal of Experimental Psychology, 61 1, 869–882. DOI logoGoogle Scholar
McBride, Dawn M., Coane, Jennifer H. & Raulerson III, Bascom A. (2006). An investigation of false memory in perceptual implicit tasks. Acta Psychologica, 123 1, 240–260. DOI logoGoogle Scholar
McNeil, Nicole M., Alibali, Martha W., & Evans, Julia L. (2000). The role of gesture in children’s comprehension of spoken language: Now they need it, now they don’t. Journal of Nonverbal Behavior, 24 1, 131–150. DOI logoGoogle Scholar
McNeill, David. (2003). Aspects of aspects. Gesture, 3 1, 1–17. DOI logoGoogle Scholar
. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Google Scholar
McNeill, David, Cassell, Justine, & McCullough, Karl-Erik. (1994). Communicative effects of speech-mismatched gestures. Research on Language and Social Interaction, 27 1, 223–237. DOI logoGoogle Scholar
Müller, Cornelia. (2013). Gestural modes of representation as techniques of depiction. In C. Müller, A. J. Cienki, E. Fricke, S. H. Ladewig, D. McNeill, & S. Teßendorf (Eds.), Body – language – communication: An international handbook on multimodality in human interaction (pp. 1687–1701). Berlin: De Gruyter Mouton.Google Scholar
Mumford, Katherine H. & Kita, Sotaro. (2014). Children use gesture to interpret novel verb meanings. Child Development, 85 1, 1181–1189. DOI logoGoogle Scholar
Odegard, Timothy N. & Lampinen, James M. (2005). Recollection rejection: Gist cuing of verbatim memory. Memory and Cognition, 33 1, 1422–1430. DOI logoGoogle Scholar
Ortega, Gerardo & Özyürek, Aslı. (2020). Systematic mappings between semantic categories and types of iconic representations in the manual modality: A normed database of silent gesture. Behavior Research Methods, 52 1, 51–67. DOI logoGoogle Scholar
Özçaliskan, Şeyda, Gentner, Dedre, & Goldin-Meadow, Susan. (2014). Do iconic gestures pave the way for children’s early verbs? Applied Psycholinguistics, 35 1, 1143–1162. DOI logoGoogle Scholar
Özyürek, Aslı, Willems, Roel M., Kita, Sotaro, & Hagoort, Peter. (2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 19 1, 605–616. DOI logoGoogle Scholar
Padden, Carol, Hwang, So-One, Lepic, Ryan, & Seegers, Sharon. (2015). Tools for language: Patterned iconicity in sign language nouns and verbs. Topics in Cognitive Science, 7 1, 81–94. DOI logoGoogle Scholar
Piccin, Thomas B. & Waxman, Sandra R. (2007). Why nouns trump verbs in word learning: New evidence from children and adults in the human simulation paradigm. Language Learning and Development, 3 1, 295–323. DOI logoGoogle Scholar
Reales, José Manuel & Ballesteros, Soledad. (1999). Implicit and explicit memory for visual and haptic objects: Cross-modal priming depends on structural descriptions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25 1, 644–663.Google Scholar
Rüschemeyer, Shirley-Ann, Brass, Marcel, & Friederici, Angela D. (2007). Comprehending prehending: Neural correlates of processing verbs with motor stems. Journal of Cognitive Neuroscience, 19 1, 855–865. DOI logoGoogle Scholar
Schönpflug, Ute. (2008). Pauses in elementary school children’s verbatim and gist free recall of a story. Cognitive Development, 23 1, 385–394. DOI logoGoogle Scholar
Singer, Melissa A. & Goldin-Meadow, Susan. (2005). Children learn when their teacher’s gestures and speech differ. Psychological Science, 16 1, 85–89. DOI logoGoogle Scholar
Sowa, Timo & Wachsmuth, Ipke. (2009). A computational model for the representation and processing of shape in coverbal iconic gestures. In Kenny R. Coventry, Thora Tenbrink, & John Bateman (Eds.), Spatial language and dialogue (pp. 132–146). Oxford: Oxford University Press. DOI logoGoogle Scholar
Wang, Xiao-lei & Plotka, Raquel. (2018). Lending a helping hand for English-Chinese bilingual children’s idiom comprehension and retention: Implications for early childhood educators. Early Years, 38 (3), 298–315. DOI logoGoogle Scholar
Wesp, Richard, Hess, Jennifer, Keutmann, Donna, & Wheaton, Karen. (2001). Gestures maintain spatial imagery. The American Journal of Psychology, 114 1, 591–600. DOI logoGoogle Scholar
Westbury, Chris & Hollis, Geoff. (2019). Conceptualizing syntactic categories as semantic categories: Unifying part-of-speech identification and semantics using co-occurrence vector averaging. Behavior Research Methods, 51 (3), 1371–1398. DOI logoGoogle Scholar
Wu, Ying Choon & Coulson, Seana. (2007). How iconic gestures enhance communication: An ERP study. Brain and Language, 101 1, 234–245. DOI logoGoogle Scholar
Cited by (1)

Cited by one other publication

Lopez-Ozieblo, Renia
2024. Effects of pedagogical gestures on learning abstract grammatical concepts in young adults. Frontiers in Communication 9 DOI logo

This list is based on CrossRef data as of 4 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.