Article published in:
Review of Cognitive Linguistics
Vol. 18:2 (2020) ► pp. 289–315

Gesturing in the wild

Evidence for a flexible mental timeline

| University of Murcia
| University of Murcia
| University of Navarra
| University of Murcia

Psycholinguistic evidence shows that spatial domains are automatically activated when processing temporal expressions. Speakers conceptualize time as a straight line deployed along different axes (mostly sagittal, though also vertical). The use of the lateral axis, which cannot be lexicalized in any language, has nonetheless been attested in temporal tasks in laboratories using a variety of experiments. This leads to the question of what axes are actually at work when conceptualizating time in oral communication.

The present study examines a great number of temporal expressions, taken from television shows, noting their associated co-speech gestures. Our results show that (1) speakers overwhelmingly use the lateral axis; (2) they are not performing simple space-to-time mappings, but are using instead a “timeline”, a material anchor which is a far more complex construct and that can explain some of the intricacies and contextual variations shown in the pattern of results.

Keywords: space-to-time metaphors, co-speech gestures, time conceptualization, timelines, material anchors
Article outline
  • 1.Introduction
    • 1.1The lateral axis
    • 1.2Multimodality and the NewsScape Library of Television News
  • 2.Gesturing the timeline: A multimodal study
    • 2.1Research questions and hypotheses
    • 2.2Materials and methodology
      • 2.2.1Data collection
      • 2.2.2Filtering
      • 2.2.3Coding
  • 3.Results
    • 3.1Construal
    • 3.2Axis
    • 3.3Directionality
  • 4.Discussion
    • 4.1The lateral timeline: New evidence from spontaneous speech
    • 4.2Flexibility
  • 5.Conclusion
  • References
Published online: 04 December 2020
https://doi.org/10.1075/rcl.00061.val
References
Alibali, M. W., Bassok, M., Solomon, K. D., Syc, S. E., & Goldin-Meadow, S.
(1999) Illuminating mental representations through speech and gesture. Psychological Science, 101, 327–333. CrossrefGoogle Scholar
Baldwin, D. A.
(2002) The rise of intentional understanding: Analogies to the ontogenesis of language. In T. Givon & B. Malle (Eds.), The evolution of language out of prelanguage (pp. 285–305). Amsterdam & Philadelphia: John Benjamins. CrossrefGoogle Scholar
Barrett, L.
(2015) Beyond the brain: how body and environment shape animal and human minds. Princeton, NJ: Princeton University Press.Google Scholar
Boroditsky, L., & Gaby, A.
(2010) Remembrances of times east: Absolute spatial representations of time in an aboriginal Australian community. Psychological Science, 211, 1635–1639. 10.1177/0956797610386621. CrossrefGoogle Scholar
Boroditsky, L., Fuhrman, O., & McCormick, K.
(2011) Do English and Mandarin speakers think about time differently? Cognition, 118(1), 126–132. CrossrefGoogle Scholar
Bressem, J.
(2013) A linguistic perspective on the notation of form features in gestures. In C. Müller, A. J. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Teßendorf (Eds.), Body – language – communication. An international handbook on multimodality in human interaction (pp. 1079–1098). Berlin & New York: Mouton de Gruyter. CrossrefGoogle Scholar
Brewer, M.
(2000) Research design and issues of validity. In H. Reis & C. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 11–26). Cambridge: Cambridge University Press.Google Scholar
Calbris, G.
(1990) The semiotics of French gesture. Bloomington & Indianapolis: Indiana University Press.Google Scholar
(2011) Elements of meaning in gesture. Amsterdam & Philadelphia: John Benjamins. CrossrefGoogle Scholar
Casasanto, D.
(2009) When is a linguistic metaphor a conceptual metaphor? In V. Evans & S. Pourcel (Eds.), New directions in Cognitive Linguistics (pp. 127–145). Amsterdam & Philadelphia: John Benjamins. CrossrefGoogle Scholar
Casasanto, D., & Boroditsky, L.
(2008) Time in the mind: using space to think about time. Cognition, 1061, 579–93. CrossrefGoogle Scholar
Casasanto, D., & Bottini, R.
(2014) Mirror-reading can reverse the flow of time. Journal of Experimental Psychology: General, 143(2), 473–479. CrossrefGoogle Scholar
Casasanto, D., & Jasmin, K.
(2012) The hands of time: Temporal gestures in English speakers. Cognitive Linguistics, 23(4), 643–674. CrossrefGoogle Scholar
Cienki, A.
(1998) Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J. -P. Koenig (Ed.), Discourse and cognition: bridging the gap (pp. 189–204). Stanford, CA: CSLI Publications.Google Scholar
Cienki, A., & Müller, C.
(2008) Metaphor, gesture and thought. In R. W. Gibbs (Ed), Cambridge handbook of metaphor and thought (pp. 483–501). Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Clark, A.
(2008) Supersizing the mind: Embodiment, action and cognitive extension. Oxford: Oxford University Press. CrossrefGoogle Scholar
Clark, H. H.
(1996) Using language. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Cooperrider, K., & Nuñez, R.
(2009) Across time, across the body: transversal temporal gestures. Gesture, 91, 181–206. CrossrefGoogle Scholar
Coulson, S., & Pagán-Cánovas, C.
(2013) Understanding timelines: Conceptual metaphor and conceptual integration. Cognitive Semiotics, 5(1–2), 198–219. CrossrefGoogle Scholar
Dancygier, B.
(2012) The language of stories. Cambridge: Cambridge University Press.Google Scholar
Dancygier, B., & Sweetser, E.
(Eds.) (2012) Viewpoint in language: A multimodal perspective. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Dirven, R., & Parings, R.
(Eds.) (2002) Metaphor and metonymy in comparison and contrast. Berlin & New York: Mouton de Gruyter.Google Scholar
Engberg-Pedersen, E.
(1993) Space in Danish sign language: The semantics and morphosyntax of the use of space in a visual language. Hamburg: SIGNUM-Verlag.Google Scholar
(1999) Space and time. In J. Allwood & P. Gardenfors (Eds.), Cognitive semantics: Meaning and cognition (pp. 131–152). Amsterdam & Philadelphia: John Benjamins. CrossrefGoogle Scholar
Emmorey, K.
(2001) Space on hand: The exploitation of signing space to illustrate abstract thought. In M. Gattis (Ed.), Spatial schemas and abstract thought (pp. 147–174). Cambridge: The MIT Press.Google Scholar
Fauconnier, G.
(1997) Mappings in thought and language. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Fauconnier, G., & Turner, M.
(1998) Conceptual integration networks. Cognitive Science, 22(2), 133–187. CrossrefGoogle Scholar
(2002) The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books.Google Scholar
Fillmore, C. J., Kay, P., & O’Connor, M.
(1988) Regularity and idiomaticity in grammatical constructions: The case of let alone. Language, 64(3), 501–538. CrossrefGoogle Scholar
Fuhrman, O., & Boroditsky, L.
(2010) Cross-cultural differences in mental representations of time: Evidence from an implicit nonlinguistic task. Cognitive Science, 341, 1430–1451. CrossrefGoogle Scholar
Goldberg, A.
(1995) Constructions: A Construction Grammar approach to argument structure. Chicago: University of Chicago Press.Google Scholar
(2006) Constructions at work. Oxford: Oxford University Press.Google Scholar
Goldin-Meadow, S., & Alibali, M. W.
(2013) Gesture’s role in speaking, learning, and creating language. Annual Review of Psychology, 1231, 448–453. CrossrefGoogle Scholar
Goldin-Meadow, S.
(2003) Gesture. Encyclopedia of cognitive science, volume 21 (pp. 289–293). London: Macmillan Reference Ltd.Google Scholar
Hutchins, E.
(2005) Material anchors for conceptual blends. Journal of Pragmatics, 371, 1555–1577. CrossrefGoogle Scholar
Ishihara, M., Keller, P. E., Rossetti, Y. & Prinz, W.
(2008) Horizontal spatial representations of time: Evidence for the STEARC effect. Cortex, 44(4), 454–461. CrossrefGoogle Scholar
Jewitt, C.
(Ed.) (2009) The Routledge handbook of multimodal analysis. London: Routledge.Google Scholar
Kendon, A.
(1995) Gestures as illocutionary and discourse structure markers in Southern Italian conversation. Journal of Pragmatics, 23(3), 247–279. CrossrefGoogle Scholar
(2004) Gesture: Visible action as utterance. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Kita, S.
(2009) Cross-cultural variation of speech-accompanying gesture: a review. Language and Cognitive Processes, 24(2), 145–167. CrossrefGoogle Scholar
Kövecses, Z.
(2005) Metaphor in culture. Universality and variation. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Lakoff, G.
(1993) The contemporary theory of metaphor. In A. Ortony (Ed.), Metaphor and thought (pp. 202–251). Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Lakoff, G., & Johnson, M.
(1980) Metaphors we live by. Chicago: University of Chicago Press.Google Scholar
(1999) Philosophy in the flesh. New York: Basic Books.Google Scholar
Le Guen, O., & Pool Balam, L. I.
(2012) No metaphorical timeline in gesture and cognition among Yucatec Mayas. Frontiers in Cultural Psychology, 31, Article 271. CrossrefGoogle Scholar
Leonard, T., & Cummins, F.
(2011) The temporal relation between beat gestures and speech. Language and Cognitive Processes, 26(10), 1457–1471. CrossrefGoogle Scholar
Levinson, S. C.
(1996) Frames of reference and Molyneux’s question: Cross-linguistic evidence. In P. Bloom, M. Peterson, L. Nadel & M. Garrett (Eds.), Language and space (pp. 109–169). Cambridge, MA: MIT Press.Google Scholar
McNeill, D.
(1992) Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Google Scholar
(2005) Gesture and thought. Chicago: University of Chicago Press. CrossrefGoogle Scholar
(2013) Gesture as a window onto mind and brain, and the relationship to linguistic creativity and ontogenesis. In C. Müller, A. J. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Teßendorf (Eds.), Body – language – communication. An international handbook on multimodality in human interaction. Volume I (pp. 28–54). Berlin & New York: Mouton de Gruyter.Google Scholar
Müller, C., Cienki, A., Fricke, E., Ladewig, S. H., McNeill, D., & Bressem, J.
(Eds.) (2013) Body – language – communication. An international handbook on multimodality in human interaction. Volume II. Berlin & New York: Mouton de Gruyter.Google Scholar
(Eds.) (2014) Body – language – communication. An international handbook on multimodality in human interaction. Volume II. Berlin & New York: Mouton de Gruyter.Google Scholar
Núñez, R., & Sweetser, E.
(2006) With the future behind them: Convergent evidence from Aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 301, 401–450. CrossrefGoogle Scholar
Ouellet, M., Santiago, J., Israeli, Z., & Gabay, S.
(2010) Is the future the right time? Experimental Psychology, 571, 308–314. CrossrefGoogle Scholar
Pagán Cánovas, C., Valenzuela, J., & Santiago, J.
(2015) Like the machete the snake: Integration of topic and vehicle in poetry comprehension reveals meaning construction processes. Psychology of Aesthetics, Creativity, and the Arts, 9(4), 385–393. CrossrefGoogle Scholar
Pagán-Cánovas, C., & Valenzuela, J.
(2017) Timelines and multimodal constructions: facing new challenges. Linguistics Vanguard, 3(1). CrossrefGoogle Scholar
Pagán-Cánovas, C., Valenzuela, J., Alcaraz-Carrión, D., Olza, I., & Ramscar, M.
(2020) Quantifying the speech-gesture relation with massive multimodal datasets: Informativity in time expressions. PLoS ONE, 15(6), e0233892. CrossrefGoogle Scholar
Radden, G.
(2004) The metaphor TIME AS SPACE across languages. In N. Baumgarten, C. Böttger, M. Motz & J. Probst (Eds.), Übersetzen, interkulturelle Kommunikation, Spracherwerb und Sprachvermittlung – Das Leben mit Mehreren Sprachen: Festschrift für Juliane House zum 60 Geburtstag (pp. 225–238). Bochum, Germany: Aks-Verlag.Google Scholar
Richter, N.
(2014) Structuring discourse: Observations on prosody and gesture in Russian TV-discourse. In C. Müller, A. J. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Teßendorf (Eds.), Body – language – communication. An international handbook on multimodality in human interaction. Volume II (pp. 1392–1400). Berlin & New York: Mouton de Gruyter. CrossrefGoogle Scholar
Santiago, J., Lupiáñez, J., Perez, E., & Funes, M. J.
(2007) Time (also) flies from left to right. Psychonomic Bulletin & Review, 14(3), 512–516. CrossrefGoogle Scholar
Santiago, J., Román, A., & Ouellet, M.
(2011) Flexible foundations of abstract thought: A review and a theory. Spatial Dimensions of Social Thought, 41–110. CrossrefGoogle Scholar
Scott, A.
(1989) The vertical dimension and time in Mandarin. Australian Journal of Linguistics, 91, 295–331. CrossrefGoogle Scholar
Sell, A. J., & Kaschak, M. P.
(2011) Processing time shifts affects the execution of motor responses. Brain & Language, 1171, 39–44. CrossrefGoogle Scholar
Sinha, C., Silva Sinha, V. da, Zinken, J., & Sampaio, W.
(2011) When time is not space: The social and linguistic construction of time intervals and temporal event relations in an Amazonian culture. Language and Cognition, 3(1), 137–169. CrossrefGoogle Scholar
Steen, F., & Turner, M.
(2013) Multimodal Construction Grammar. In M. Borkent, B. Dancygier, & J. Hinnell (Eds.), Language and the creative mind (pp. 255–274). Stanford: CSLI Publications.Google Scholar
Stockwell, P.
(1999) The inflexibility of invariance. Language and Literature, 8(2), 125–142. CrossrefGoogle Scholar
Torralbo, A., Santiago, J., & Lupiáñez, J.
(2006) Flexible conceptual projection of time onto spatial frames of reference. Cognitive Science, 30(4), 745–757. CrossrefGoogle Scholar
Turner, M.
(1996) The literary mind. New York: Oxford University Press US.Google Scholar
(2014) The origin of ideas: blending, creativity, and the human spark. New York: Oxford University Press US.Google Scholar
Ulrich, R., Eikmeier, V., Vega, I., Ruiz Fernández, S., Alex-Ruf, S., & Maienborn, C.
(2012) With the past behind and the future ahead: Back-to-front representation of past and future sentences. Memory & Cognition, 40(3), 483–495. CrossrefGoogle Scholar
Ulrich, R., & Maienborn, C.
(2010) Left–right coding of past and future in language: The mental timeline during sentence processing. Cognition, 1171, 126–138. CrossrefGoogle Scholar
Valenzuela, J., Hilferty, J., & Garachana, M.
(2005) On the reality of grammatical constructions. Annual Review of Cognitive Linguistics, 31, 201–215. CrossrefGoogle Scholar
Walker, E., & Cooperrider, K.
(2015) The continuity of metaphor: evidence from temporal gestures. Cognitive Science, 40(2), 481–495. CrossrefGoogle Scholar
Weger, U. W., & Jay, P.
(2008) Time flies like an arrow: Space–time compatibility effects suggest the use of a mental timeline. Psychonomic Bulletin and Review, 15(2), 426–430. CrossrefGoogle Scholar
Wang, L., & Chu, M.
(2013) The role of beat gesture and pitch accent in semantic processing: an ERP study. Neuropsychologia, 51(13), 2847–55. CrossrefGoogle Scholar
Zhang, J., & Patel, V. L.
(2006) Distributed cognition, representation, and affordance. Pragmatics & Cognition, 14(2), 333–341. CrossrefGoogle Scholar
Zima, E.
(2014) Gibt es multimodale Konstruktionen? Eine Studie zu [V(motion) in circles] und [all the way from X PREP Y]. Gesprächsforschung – Online-Zeitschrift zur verbalen Interaktion, 151, 1–48.Google Scholar
Cited by

Cited by 2 other publications

Alcaraz Carrión, Daniel, Cristóbal Pagán Cánovas & Javier Valenzuela
2020. Enaction Through Co-speech Gesture: The Rhetorical Handing of the Mental Timeline. Zeitschrift für Anglistik und Amerikanistik 68:4 ► pp. 411 ff. Crossref logo
Valenzuela, Javier & Daniel Alcaraz Carrión
2020. Temporal Expressions in English and Spanish: Influence of Typology and Metaphorical Construal. Frontiers in Psychology 11 Crossref logo

This list is based on CrossRef data as of 27 april 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.