Cover not available
Part of
Time Representations in the Perspective of Human Creativity
Edited by Anna Piata, Adriana Gordejuela and Daniel Alcaraz Carrión
[Human Cognitive Processing 75] 2022
► pp. 187–208

Chapter 9
Physical and imaginary landmarks in English time gestures

| University of Wisconsin-Madison

In this chapter I discuss several creative instances of time co-speech gestures in English obtained from a repository of television news. I argue that, even though time gestures tend to follow certain patterns in their form, speakers may deviate, giving rise to more creative instances with a more complex meaning. The chapter begins with a brief introduction to the relation between speech and gesture, with a focus on temporal cognition, and timelines. Then, I discuss the notion of creativity in gesture research. I analyze five examples of time gestures that fall under three main categories: the ad hoc creation of imaginary timelines, the inclusion of physical items to the time construal and the emotional and pragmatic component of facial gestures.

Keywords: material anchor, spatial metaphor, television news, timelines
Article outline
  • 1.Introduction
    • 1.1What is a gesture?
    • 1.2Time gestures
  • 2.Creativity in (temporal) gestures
    • 2.1Creation and manipulation of imaginary timelines
    • 2.2Inclusion of physical items
    • 2.3Facial expression and time gestures
  • 3.Conclusion
  • Notes
  • References
Published online: 10 November 2022
DOI logo
https://doi.org/10.1075/hcp.75.09alc
References
Alcaraz Carrión, D.
(2019) Time gestures: Time conceptualisation in English with evidence from gestures in a multimodal corpus. PhD dissertation, Lancaster University, UK. DOI logoGoogle Scholar
Beattie, G., & Shovelton, H.
(1999) Mapping the range of information contained in the iconic hand gestures that accompany spontaneous speech. Journal of Language and Social Psychology, 18(4), 438–462. DOI logoGoogle Scholar
Beattie, G., Webster, K., & Ross, J.
(2010) The fixation and processing of the iconic gestures that accompany talk. Journal of Language and Social Psychology, 29(2), 194–213. DOI logoGoogle Scholar
Bottini, R., Crepaldi, D., Casasanto, D., Crollen, V., & Collignon, O.
(2015) Space and time in the sighted and blind. Cognition, 141, 67–72. DOI logoGoogle Scholar
Busso, C., & Narayanan, S.
(2007) Interrelation between speech and facial gestures in emotional utterances: A single subject study. Audio, Speech, and Language Processing, 15, 2331–2347. DOI logoGoogle Scholar
Calbris, G.
(1990) The semiotics of French gestures. Bloomington: Indiana University Press.Google Scholar
Casasanto, D., & Bottini, R.
(2010) Can mirror-reading reverse the flow of time? In T. S. Hölscher (Ed), Spatial cognition, vol. II (pp. 1342–1347). Berlin Heidelberg: Springer. DOI logoGoogle Scholar
Casasanto, D., & Jasmin, K.
(2012) The hands of time: Temporal gestures in English speakers. Cognitive Linguistics, 23(4), 643–674. DOI logoGoogle Scholar
Cienki, A.
(1998) Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J.-P. Koenig (Ed.), Discourse and cognition: Bridging the gap (pp. 189–204). Stanford, CA: Center for the Study of Language and Information.Google Scholar
(2005) Gesture and the question of literal versus non-literal reference. In B. Lewandowska-Tomaszczyk (Ed.), The literal and nonliteral in language and thought (pp. 281–298). Bern: Peter Lang.Google Scholar
(2008) Why study metaphor and gesture? In A. Cienki (Ed.), Metaphor and gesture (pp. 5–25) Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Cienki, A., & Mittelberg, I.
(2013) Creativity in the forms and functions of spontaneous gesture with speech. In T. Veale, K. Feyaerts & C. Forceville (Eds.), The agile mind: Multidisciplinary study of a multifaceted phenomenon (pp. 231–252). Berlin: Mouton de Gruyter. DOI logoGoogle Scholar
Cooperrider, K., & Núñez, R.
(2009) Across time, across the body: Transversal temporal gestures. Gesture, 9(2), 181–206. DOI logoGoogle Scholar
Coulson, S., & Pagán Cánovas, C.
(2014) Understanding timelines: Conceptual metaphor and conceptual integration. Journal of Cognitive Semiotics, 1–2, 198–219.Google Scholar
Davis, S. B.
(2012) History on the line: Time as dimension. Design Issues, 28, 4–17. DOI logoGoogle Scholar
Di Bono, M., Casarotti, M., Prifitis, K., Gava, L., Umiltà, C., & Zorzi, M.
(2012) Priming the mental timeline. Journal of Experimental Psychology: Human Perception and Performance, 38(4), 838–842.Google Scholar
Ekman, P., & Friesen, W. V.
(1969) The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1, 49–98. DOI logoGoogle Scholar
Ekman, P., Sorenson, E. R., & Friesen, W. V.
(1969) Pan-cultural elements in facial displays of emotion. Science, 164(3875), 86–88. DOI logoGoogle Scholar
Emmorey, K.
(2001) Space on hand: The exploitation of signing space to illustrate abstract thought. In M. Gattis (Ed.), Spatial schemas and abstract thought (pp. 147–174). Cambridge, MA: MIT Press.Google Scholar
Engberg-Pedersen, E.
(1999) Space and time. In J. Allwood & P. Gärdenfors (Eds.), Cognitive semantics: Meaning and cognition (pp. 131–152). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Evans, V.
(2004) The structure of time. Amsterdam: John Benjamins. DOI logoGoogle Scholar
(2013) Language and time: A cognitive linguistics approach. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Floyd, S.
(2016) Modally hybrid grammar?: Celestial pointing for time-of-day reference in Nheengatú. Language, 92(1), 31–64. DOI logoGoogle Scholar
Fuhrman, O., McCormick, K., Chen, E., Jiang, H., Shu, D., Mao, S., & Boroditsky, L.
(2011) Cross-cultural differences in mental representations of time: Evidence from an implicit non-linguistic task. Cognitive Science, 35, 1305–1328. DOI logoGoogle Scholar
Galton, A.
(2011) Time flies but space does not: Limits to the spatialization of time. Journal of Pragmatics, 43(3), 695–703. DOI logoGoogle Scholar
Gu, Y.
(2018) Chinese hands of time: The effect of language and culture on temporal gestures and spatio-temporal reasoning. Utrecht: LOT Publisher.Google Scholar
Gu, Y., Mol, L., Hoetjes, M. W., & Swerts, M. G. J.
(2013) What can Chinese speakers’ temporal gestures reveal about their conception of time? In Proceedings of TiGeR 2013. Available at: [URL]Google Scholar
Gu, Y., Mol, L., Hoetjes, M., & Swerts, M.
(2017) Conceptual and lexical effects on gestures: The case of vertical spatial metaphors for time in Chinese. Language, Cognition and Neuroscience, 32(8), 1048–1063. DOI logoGoogle Scholar
Gullberg, M., & Holmqvist, K.
(2002) Visual attention towards gestures in face-to-face interaction vs. on screen. In I. Wachsmuth & T. Sowa (Eds.), Gesture and sign language in human-computer interaction (pp. 206–214). Berlin: Springer. DOI logoGoogle Scholar
Hartmann, M., & Mast, F.
(2012) Moving along the mental time line influences the processing of future related words. Consciousness and Cognition, 21, 1558–1562. DOI logoGoogle Scholar
Hartmann, M., Martarelli, C. S., Mast, F. W., & Stocker, K.
(2014) Eye movements during mental time travel follow a diagonal line. Consciousness and Cognition, 30, 201–209. DOI logoGoogle Scholar
Hutchins, E.
(2005) Material anchors for conceptual blends. Journal of Pragmatics, 37, 1555–1577. DOI logoGoogle Scholar
Ishihara, M., Keller, P., Rossetti, Y., & Prinz, W.
(2008) Horizontal spatial representations of time: Evidence for the STEARC effect. Cortex, 44(4), 454–461. DOI logoGoogle Scholar
Izard, C. E.
(1971) The face of emotion. New York: Appleton-Century-Crofts.Google Scholar
Kendon, A.
(1980) Gesture and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), Nonverbal communication and language (pp. 207–227). Berlin: Mouton de Gruyter.Google Scholar
(1988) How gestures can become like words. In F. Poyatos (Ed.), Cross-cultural perspectives in nonverbal communication (pp. 131–141). Boston: Hogrefe & Huber Publishers.Google Scholar
(2004) Gesture: Visible action as utterance. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Kidwell, M.
(2013) Framing, grounding and coordinating conversational interaction: Posture, gaze, facial expression and movement in space. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Teßendorf (Eds.), Body-language-communication: An international handbook on multimodality in human interaction, vol. I (pp. 100–112). Berlin: Mouton de Gruyter. DOI logoGoogle Scholar
Le Guen, O., & Pool Balam, L. I.
(2012) No metaphorical timeline in gesture and cognition among Yucatec Mayas. Frontiers in Psychology, 3, 271. DOI logoGoogle Scholar
Mancini, G., Biolcati, R., Agnoli, S., Andrei, F., & Trombini, E.
(2018) Recognition of facial emotional expressions among Italian pre-adolescents, and their affective reactions. Frontiers in Psychology, 9, 1303. DOI logoGoogle Scholar
Matsumoto, D., Yoo, S. H., Fontaine, J., Anguas-Wong, A., Arriola, M., Ataca, B., Bond, M., Boratav, H., Breugelmans, S., Cabecinhas, R., Chae, J., Chin, W., Comunian, A., DeGere, D., Bley, L., Djunaidi, A., Fok, H., Friedlmeier, W., Ghosh, A. & Grossi, E.
(2008) Mapping expressive differences around the world: The relationship between emotional display rules and individualism versus collectivism. Journal of Cross-Cultural Psychology, 39, 55–74. DOI logoGoogle Scholar
McNeill, D.
(1992) Hand and mind. Chicago: University of Chicago Press.Google Scholar
(2005) Gesture and thought. Chicago: University of Chicago Press. DOI logoGoogle Scholar
Miles, L., Karpinska, K., Lumsden, J., & Macrae, C. N.
(2010a) The meandering mind: Vection and mental time travel. PLOS ONE, 5(5), e10825. DOI logoGoogle Scholar
Miles, L., Nind, L., & Macrae, C.
(2010b) Moving through time. Psychological Science, 21(2), 222–223. DOI logoGoogle Scholar
Moore, K.
(2000) Spatial experience and temporal metaphors in Wolof: Point of view, conceptual mappings, and linguistic practice. PhD dissertation, University of California at Berkeley, US.
(2006) Space-to-time mappings and temporal concepts. Cognitive Linguistics, 17(2), 199–244. DOI logoGoogle Scholar
(2011) Ego-perspective and field-based frames of reference: Temporal meanings of FRONT in Japanese, Wolof and Aymara. Journal of Pragmatics, 43(3), 759–776. DOI logoGoogle Scholar
(2014) The spatial language of time metaphor, metonymy and frames of reference. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Müller, D., & Schwarz, W.
(2008) “1-2-3”: Is there a temporal number line? Evidence from a serial comparison task. Experimental Psychology, 55(3), 143–150. DOI logoGoogle Scholar
Núñez, R., Cooperrider, K., Doan, D., & Wassmann, J.
(2012) Contours of time: Topographic construals of past, present, and future in the Yupno valley of Papua New Guinea. Cognition, 124(1), 25–35. DOI logoGoogle Scholar
Nilsson, A. L.
(2016) Embodying metaphors: Signed language interpreters at work. Cognitive Linguistics, 27(1), 35–65. DOI logoGoogle Scholar
Núñez, R., & Sweetser, E.
(2006) With the future behind them: Convergent evidence from Aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30(3), 1–49. DOI logoGoogle Scholar
Ouellet, M., Santiago, J., Israeli, Z., & Gabay, S.
(2010) Is future the right time? Experimental Psychology, 57(4), 308–314. DOI logoGoogle Scholar
Özkul, S., Bozkurt, E., Asta, S., Yemez, Y., & Erzin, E.
(2012) Multimodal analysis of upper-body gestures, facial expressions and speech. In L. Devillers et al. (Eds.), Proceedings of the 4th International Workshop on Corpora for Research on Emotion Sentiment and Social Signals (pp. 77–81). Available at: [URL]Google Scholar
Pagán Cánovas, C., & Valenzuela, J.
(2017) Timelines and multimodal constructions: Facing new challenges. Linguistic Vanguard, 3(1), 20160087. DOI logoGoogle Scholar
Pagán Cánovas, C., Valenzuela, J., Alcaraz Carrión, D., Olza, I., & Ramscar, M.
(2020) Quantifying the speech-gesture relation with massive multimodal datasets: Informativity in time expressions. PLOS ONE, 15(6), e0233892. DOI logoGoogle Scholar
Parril, F., & Sweetser, E.
(2004) What we mean by meaning: Conceptual integration in gesture analysis and transcription. Gesture, 4(2), 197–219. DOI logoGoogle Scholar
Ricci Bitti, P. R.
(2013) Facial expression and social interaction. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Teßendorf (Eds.), Body-language-communication: An international handbook on multimodality in human interaction, vol. II (pp. 1342–1349). Berlin: Mouton de Gruyter.Google Scholar
Rinn, W.
(1984) The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expressions. Psychological Bulletin, 95(1), 52–77. DOI logoGoogle Scholar
Rives Bogart, K., Cole, J., & Briegel, W.
(2013) On the consequences of living without facial expression. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Teßendorf (Eds.), Body-language-communication: An international handbook on multimodality in human interaction, vol. II (pp. 1969–1982). Berlin: Mouton de Gruyter.Google Scholar
Santiago, J., Román, A., Ouellet, M., Rodríguez, N., & Pérez-Azor, P.
(2008) In hindsight, life flows from left to right. Psychological Research, 74(1), 59–70. DOI logoGoogle Scholar
Santiago, J., Lupiáñez, J., Pérez, E., & Funes, M.
(2007) Time also flies from left to right. Psychonomic Bulletin & Review, 14(3), 512–516. DOI logoGoogle Scholar
Schermer, T., & Koolhof, C.
(1990) The reality of time-lines: Aspects of tense in Sign Language of the Netherlands (SLN). In S. Prillwitz & T. Vollhaber (Eds.), Current trends in European sign language research (pp. 295–305). Hamburg: Signum.Google Scholar
Stanzione, C., Perez, S. M., & Lederberg, A.
(2013) Assessing aspects of creativity in deaf and hearing high school students, The Journal of Deaf Studies and Deaf Education, 18(2), 228–241. DOI logoGoogle Scholar
Susskind, J. M., & Anderson, A. K.
(2008) Facial expression form and function. Communicative & Integrative Biology, 1(2), 148–149. DOI logoGoogle Scholar
Sutton-Spence, R.
(2017) Constructing visual images in creative sign language in solo and collaborative performance: Space and embodiment. African Studies, 76(3), 360–380, DOI logoGoogle Scholar
Tomkins, S. S.
(1962) Affect, imagery, consciousness, vol. I: The positive affects. New York: Springer.Google Scholar
Tompkins, S. S.
(1963) Affect, imagery, consciousness, vol. II: The negative affects. New York: Springer.Google Scholar
Torralbo, A., Santiago, J., & Lupáñez, J.
(2006) Flexible conceptual projection of time onto spatial frames of reference. Cognitive Science, 30(4), 745–757. DOI logoGoogle Scholar
Tversky, B., Kugelmass, S., & Winter, A.
(1991) Cross-cultural and developmental trends in graphic productions. Cognitive Psychology, 23 (4), 515–557. DOI logoGoogle Scholar
Valenzuela, J., & Alcaraz Carrión, D.
(2020) Temporal expressions in English and Spanish: Influence of typology and metaphorical construal. Frontiers in Psychology, 11, 543933. DOI logoGoogle Scholar
Valenzuela, J., Pagán Cánovas, C., Olza, I., & Alcaraz Carrión, D.
(2020) Gesturing in the wild: Spontaneous gestures co-occurring with temporal demarcative expressions provide evidence for a flexible mental timeline. Review of Cognitive Linguistics, 18(2), 289–316. DOI logoGoogle Scholar
Vallesi, A., Binns, M., & Shallice, T.
(2008) An effect of spatial-temporal association of response codes: Understanding the cognitive representations of time. Cognition, 107(2), 501–527. DOI logoGoogle Scholar
Vallesi, A., Mclntosh, A., & Stuss, D.
(2011) How time modulates spatial responses. Cortex, 47, 148–156. DOI logoGoogle Scholar
Walker, E., Bergen, B., & Núñez, R.
(2013) Investigating spatial axis recruitment in temporal reckoning through acoustic stimuli and non-spatial responses. UC San Diego CRL Technical Report, 25, 1–10.Google Scholar
(2014) Disentangling spatial metaphors for time using non-spatial responses and auditory stimuli. Metaphor & Symbol, 29, 316–327. DOI logoGoogle Scholar
Walker, E., & Cooperrider, K.
(2016) The continuity of metaphor: Evidence from temporal gestures. Cognitive Science, 40(2), 481–495. DOI logoGoogle Scholar
Weger, U., & Pratt, J.
(2008) Time flies like an arrow: Space-time compatibility effects suggest the use of a mental timeline. Psychonomic Bulletin & Review, 15(2), 426–430. DOI logoGoogle Scholar
Wilcox, S.
(2002) The iconic mapping of space and time in signed languages. In L. Albertazzi (Ed.), Unfolding perceptual continua (pp. 255–281). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Williams, R.
(2008) Gesture as a conceptual mapping tool. In A. Cienki & C. Müller (Eds.), Metaphor and gesture (pp. 55–92). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Zorzi, M., Bonato, M., Treccani, B., Scalambrin, G., Marenzi, R., & Priftis, K.
(2012) Neglect impairs explicit access to the mental number line. Frontiers in Human Neuro-science, 6(125), 1–12.Google Scholar