This paper lays out the foundations of a frame-based account of gesture pragmatics through detailing how frames and metonymy interact not only in motivating gestural sign formation, but also in guiding crossmodal processes of pragmatic inferencing. It is argued that gestures recruiting frame structures tend to profile deeply embodied, routinized aspects of scenes (in the Fillmorian sense of the term), that is, of the motivating context of frames. Two kinds of embodied frame structures situated at different levels of abstraction, schematicity, and entrechment are proposed: (A) Basic physical action and object frames understood as directly experientially grounded frames involving physical action and interaction with the material and social world; (B) Complex, highly abstract frame structures that are more detached from the motivating contexts of experience. It is further suggested that gestures exhibiting a comparably low degree of iconicity and/or indexicality are likely to assume pragmatic rather than referential functions. Finally, potential avenues for further research into the relation of scenes, frames, and (multimodal) constructions are outlined.
(2012) Repetitions in gesture: Structures, functions, and cognitive aspects. Ph.D. dissertation, European University Viadrina, Frankfurt (Oder).
Bressem, Jana & Cornelia Müller
(2014) The family of AWAY-gestures. In: Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction, Volume 21 (pp. 1592–1604). Berlin & Boston: De Gruyter Mouton.
Calbris, Geneviève
(1990) The semiotics of French gestures. Cambridge: Cambridge University Press.
(1998) Metaphoric gestures and some of their relations to verbal metaphoric expressions. In John-Pierre Koenig (Ed.), Discourse and cognition: Bridging the gap (pp. 189–204). Stanford: CSLI Publications.
Cienki, Alan
(2013a) Cognitive Linguistics: Spoken language and gesture as expressions of conceptualization. In Cornelia Müller, Alan Cienki, Elan Fricke, Silvia H. Ladewig, David McNeill, & Sedinha Tessendorf (Eds.), Body – language – communication: An international handbook on multimodality in human interaction, Volume 11 (pp. 182–201). Berlin & Boston: Mouton de Gruyter.
(1996) Using language. Cambridge: Cambridge University Press.
Clark, Herbert H.
(2003): Pointing and placing. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 243–268). Mahwah, N.J. & London: Lawrence Erlbaum Associates.
Coulson, Seana
(2001) Semantic leaps: Frame-shifting and conceptual blending in meaning construction. Cambridge: Cambridge University Press.
Croft, William
(1993) The role of domains in the interpretation of metaphor and metonymy. Cognitive Linguistics, 41, 335–370.
Croft, William & Alan A. Cruse
(2004) Cognitive linguistics. Cambridge: Cambridge University Press.
Dancygier, Barbara & Eve E. Sweetser
(2005) Mental spaces in grammar: Conditional constructions. Cambridge: Cambridge University Press.
Dancygier, Barbara & Eve E. Sweetser
(2012) Viewpoint in language: A multimodal perspective. Cambridge: Cambridge University Press.
Dancygier, Barbara & Eve E. Sweetser
(2014) Figurative language. Cambridge: Cambridge University Press.
Debras, Camille
(2014) L’Expression multimodale du positionnement interactionnel (multimodal stance-taking). Unpublished doctoral dissertation, Université Sorbonne Nouvelle – Paris 3.
Dirven, René & Ralf Pörings
(Eds.) (2002) Metaphor and metonymy in comparison and contrast. Berlin & New York: Mouton de Gruyter.
Enfield, Nick J.
(2011) Elements of formulation. In Jürgen Streeck, Charles Goodwin, & Curtis LeBaron (Eds.), Embodied interaction: Language and the body in the material world (pp. 59–66). Cambridge: Cambridge University Press.
(1975) An alternative to checklist theories of meaning. In Cathy Cogen, Henry Thompson, Graham Thurgood, Kenneth Whistler, & James Wright (Eds.), Proceedings of the first annual meeting of the Berkeley Linguistics Society (pp. 123–131). Berkeley, CA: Berkeley Linguistics Society.
Fillmore, Charles J.
(1977) Scenes-and-frames semantics. In Antonio Zampolli (Ed.), Linguistic structures processing, Volume 41 (pp. 55–81). Amsterdam & New York: North Holland.
Fillmore, Charles J.
(1982) Frame semantics. In Linguistic Society of Korea (Ed.), Linguistics in the morning calm (pp. 111–137). Seoul: Hanshin.
Fillmore, Charles J.
(1985) Frames and the semantics of understanding. Quarderni di Semantica, 61, 222–254.
Fillmore, Charles J.
(1988) The mechanisms of construction grammar. Berkeley Linguistics Society, 141, 35–55.
Fillmore, Charles J. & Collin F. Baker
(2009) A frames approach to semantic analysis. In Bernd Heine & Heiko Narrog (Eds.), The Oxford handbook of linguistic analysis (pp. 313–340). Oxford: Oxford University Press.
Fricke, Ellen
(2012) Grammatik multimodal: Wie Wörter und Gesten zusammenwirken. Berlin & New York: Mouton de Gruyter.
Gibbs, Jr., Raymond W.
(1994) The poetics of mind: Figurative thought, language, and understanding. Cambridge: Cambridge University Press.
Gibbs, Jr., Raymond W.
(1999) Speaking and thinking with metonymy. In Günter Radden & Zoltan Kövecses (Eds.) Metonymy in language and thought (pp. 61–76). Amsterdam & Philadelphia: John Benjamins.
Gibbs, Jr., Raymond W.
(2006) Embodiment and cognitive science. Cambridge: Cambridge University Press.
Gibbs, Jr., Raymond W.
2017). The embodied and discourse view of metaphor: Why these are not so different and how they can be brought closer together. In Beate Hampe Ed. Metaphor: Embodied cognition and discourse pp. 319 334 Cambridge Cambridge University Press
Goldberg, Adele
(1995) Constructions: A construction grammar approach to argument structure. Chicago: The University of Chicago Press.
Goodwin, Charles
(2007) Environmentally coupled gestures. In Susan Duncan, Justine Cassell, & Elena Levy (Eds.), Gesture and the dynamic dimensions of language (pp. 195–212). Amsterdam & Philadelphia: John Benjamins.
Goodwin, Charles
(2011) Contextures of action. In Jürgen Streeck, Charles Goodwin, & Curtis LeBaron (Eds.), Embodied interaction: Language and body in the material world (pp. 182–193). Cambridge: Cambridge University Press.
Grady, Joe
(1997) Foundations of meaning: primary metaphors and primary scenes. Doctoral dissertation, University of California at Berkeley.
Grady, Joe & Christopher Johnson
(2002) Converging evidence for the notions of subscene and primary scene. In René Dirven & Ralph Pörings (Eds.), Metaphor and metonymy in comparison and contrast (pp. 533–554). Berlin: Mouton de Gruyter.
Grandhi, Sukeshini A., Gina Joue, & Irene Mittelberg
(2011) Understanding naturalness and intuitiveness in gesture production: Insights for touchless gestural interfaces. Proceedings of the ACM2011 conference on Human Factors in Computing Systems (CHI), Vancouver, B.C.
Hassemer, Julius
(2016) Towards a theory of Gesture Form Analysis: Imaginary forms as part of gesture conceptualisation with empirical support from motion‐capture data. Doctoral dissertation, RWTH Aachen University.
Hopper, Paul & Sandra Thompson
(1980) Transitivity in grammar and discourse. Language, 561, 251–299.
(1956/1990) Two aspects of language and two types of aphasic disturbances. In Linda R. Waugh & Monique Monville-Burston (Eds.), Roman Jakobson, On language (pp. 115–133). Cambridge, MA: Harvard University Press.
Jakobson, Roman
(1960/1987) Linguistics and poetics. In Krystyna Pomorska & Stephen Rudy (Eds.), Roman Jakobson – Language in literature (pp. 62–94). Cambridge, MA: Belknap of Harvard University Press.
Jakobson, Roman & Krystyna Pomorska
(1983) Dialogues. Cambridge: Harvard University Press.
Janzen, Terry & Barbara Shaffer
(2002) Gesture as substrate in the process of ASL grammaticalization. In Richard P. Meier, Kearsy Cormier, & David Quintos-Pozos (Eds.), Modality and structure in signed and spoken language (pp. 199–223). Cambridge: Cambridge University Press.
Johnson, Mark
(1987) The body in the mind: The bodily basis of meaning, imagination, and reason. Chicago: University of Chicago Press.
Johnson, Mark
(2005) The philosophical significance of image schemas. In Beate Hampe (Ed.), From perception to meaning: Image schemas in cognitive linguistics (pp. 15–33). Berlin: Mouton de Gruyter.
Johnson, Mark
(2007) The meaning of the body: Aesthetics of human understanding. Chicago: The University of Chicago Press.).
Kendon, Adam
(1995) Gestures as illocutionary and discourse structure markers in southern Italian conversation. Journal of Pragmatics, 231, 247–279.
Kendon, Adam
(2004) Gesture: Visible action as utterance. Cambridge: Cambridge University Press.
(1972) The transformation of reality in narrative syntax. In William Labov, Language in the inner city (pp. 354–396). Philadelphia: The University of Pennsylvania Press.
Ladewig, Silva H.
(2014) Recurrent gestures. In: Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction, Volume 21 (pp. 1558–1575), Berlin & Boston: Mouton de Gruyter.
Lakoff, George
(1987) Women, fire, and dangerous things: What categories reveal about the mind. Chicago: The University of Chicago Press.
Langacker, Ronald W.
(1987) Foundations of cognitive grammar: Theoretical prerequisites, Volume 11. Stanford: Stanford University Press.
(2009) Metonymic grammar. In Klaus-Uwe Panther, Linda L. Thornburg, Antonio Barcelona (Eds.), Metonymy and metaphor in grammar (pp. 45–71). Amsterdam & Philadelphia: John Benjamins.
Littlemore, Jeanette
(2015) Metonymy: Hidden shortcuts in language, thought and communication. Cambridge: Cambridge University Press.
McNeill, David
(1992) Hand and mind: What gestures reveal about thought. Chicago: The University of Chicago Press.
McNeill, David
(2005) Gesture and thought. Chicago: The University of Chicago Press.
Mittelberg, Irene
(2006) Metaphor and metonymy in language and gesture: Discourse evidence for multimodal models of grammar (Doctoral dissertation, Cornell University). Ann Arbor, MI: UMI.
(2010a) Geometric and image-schematic patterns in gesture space. In Vyvyan Evans & Paul Chilton (Eds.), Language, cognition, and space: The state of the art and new directions (pp. 351–385). London: Equinox.
Mittelberg, Irene
(2010b) Interne und externe Metonymie: Jakobsonsche Kontiguitäts-beziehungen in redebegleitenden Gesten. Sprache und Literatur, 41 (1), 112–143.
Mittelberg, Irene
(2013a) The exbodied mind: Cognitive-semiotic principles as motivating forces in gesture. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Sedinha Tessendorf (Eds.), Body – language – communication: An international handbook on multimodality in human interaction (pp. 750–779). Berlin & Boston: Mouton de Gruyter.
Mittelberg, Irene
(2013b) Balancing acts: Image schemas and force dynamics as experiential essence in pictures by Paul Klee and their gestural enactments. In Michael Bokrent, Barbara Dancygier, & Jennifer Hinnell (Eds.), Language and the creative mind (pp. 325–346). Stanford: CSLI.
Mittelberg, Irene
(2014) Gestures and iconicity. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction (pp. 1712–1732). Berlin & Boston: Mouton de Gruyter.
Mittelberg, Irene
(2017) Multimodal existential constructions in German: Manual actions as experiential substrate for grammatical and gestural patterns. Linguistics Vanguard, 3, s1. .
Mittelberg, Irene & Linda R. Waugh
(2009) Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In Charles Forceville & Eduardo Urios-Aparisi (Eds.), Multimodal metaphor (pp. 329–356). Berlin & New York: Mouton de Gruyter.
Mittelberg, Irene & Linda R. Waugh
(2014) Gestures and metonymy. In: Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction (pp. 1747–1766). Berlin & Boston: Mouton de Gruyter.
Mittelberg, Irene & Gina Joue
(2017) Source actions ground metaphor via metonymy: Toward a frame-based account of gestural action in multimodal discourse. In Beate Hampe (Ed.), Metaphor: Embodied cognition and discourse (pp. 119–137). Cambridge: Cambridge University Press.
(2004) Forms and uses of the palm up open hand: A case of a gesture family? In Cornelia Müller & Roland Posner (Eds.), The semantics and pragmatics of everyday gesture: The Berlin conference (pp. 233–256). Berlin: Weidler Verlag.
(1960) Collected papers of Charles Sanders Peirce (1931–1958), Volume I: Principles of philosophy; Volume II: Elements of logic. Charles Hartshorne & Paul Weiss (Eds.). Cambridge, MA: The Belknap of Harvard University Press.
Pfau, Roland & Markus Steinbach
(2004) On grammaticalization: Do sign languages follow the well-trodden paths?Proceedings of the 8th Theoretical Issues in Sign Language Research (TISLR 8) Conference, Barcelona.
Peirsman, Yves & Dirk Geeraerts
(2006) Metonymy as a prototypical category. Cognitive Linguistics, 17 (3), 269–316.
Priesters, Matthias A.
(2012) Functional patterns in gesture space. Correlating location and function of gestures using 3D motion capture technology. Unpublished M.A. thesis, RWTH Aachen University.
Rizzolazzi, Giacomo & Michael A. Arbib
(1998) Language within our grasp. Trends in Neurosciences, 21 (5), 188–194.
Schank, Robert. C. & Robert Abelson
(1977) Scripts, goals, plans, and understanding. Hillsdale, NJ: Lawrence Erlbaum.
Schoonjans, Steven
(2014) Modalpartikeln als multimodale Konstruktionen. Eine korpusbasierte Kookkurrenzanalyse von Modalpartikeln und Gestik im Deutschen. Unpublished doctoral dissertation, KU Leuven.
Schoonjans, Steven, Paul Sambre, Geert Brône, & Kurt Feyaerts
(2016) Vers une analyse multimodale du sens. Perspectives constructionnelles sur la gestualité co-grammaticale. Langages, 2011, 33–50.
Slobin, Dan
(1985) Crosslinguistic evidence for the language-making capacity. In Dan Slobin (Ed.), A crosslinguistic study of language acquisition. Volume 2: Theoretical issues (pp. 1157–1256). Hillsdale, NJ: Lawrence Erlbaum.
Sperber, Dan & Deirdre Wilson
(1986/1995) Relevance: Communication and cognition (2nd Ed.), Oxford: Blackwell.
Steen, Francis & Mark Turrier
(2013) Multimodal construction grammar. In Mike Borkent, Barbara Dancygier, & Jennifer Hinnell (Eds.), Language and the creative mind (pp. 255–274), Stanford: CSLI.
(2013) Praxeology of gesture, In Cornelia Müller, Alan Cienki, Ellen Fricke, Silvia H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication. An international handbook of multimodality in human interaction, Volume 11 (pp. 674–688). Berlin & Boston: Mouton de Gruyter.
Streeck, Jürgen, Charles Goodwin, & Curtis LeBaron
(Eds.) (2011) Embodied interaction: Language and the body in the material world. Cambridge: Cambridge University Press.
Sweetser, Eve E.
(1990) From etymology to pragmatics: Metaphorical and cultural aspects of semantic structure. Cambridge: Cambridge University Press.
Sweetser, Eve E.
(1998) Regular metaphoricity in gesture: Bodily-based models of speech interaction. Actes du 16e Congrès International des Linguistes (CD-ROM), Elsevier.
(2012) Viewpoint and perspective in language and gesture: From the ground down. In Barbara Dancygier & Eve E. Sweetser (Eds.), Viewpoint in language: A multimodal perspective (pp. 1–24). Cambridge: Cambridge University Press.
(2014) Humorous implications and meanings: A multimodal study of sarcasm in interactional humor. Unpublished doctoral dissertation, Université Charles de Gaulle – Lille 3.
Talmy, Leonard
(1988) Force dynamics in language and cognition. Cognitive Science, 121, 49–100.
Talmy, Leonard
(2013) Gestures as cues to a target. Manuscript of talk presented at the ICLC 12, University of Edmonton, Alberta, June24 2013.
Waugh, Linda R. & Monique Monville-Burston
(1990) Roman Jakobson: His life, work and influence. Introduction to: Linda R. Waugh & Monique Monville Burston (Eds.), Jakobson on language (pp. 1–45). Cambridge: Belknap of Harvard University Press.
Wehling, Elisabeth
(2010) Argument is gesture war: Function, form and prosody of discourse structuring gestures in political argument. Proceedings of the 35th Annual Meeting of the Berkeley Linguistics Society (pp. 54–65). Berkeley, CA: Berkeley Linguistics Society.
(2014) Gibt es multimodale Konstruktionen? Eine Studie zu [V(motion) in circles] und [all the way from X PREP Y], Gesprächsforschung, 151, 1–48.
Zima, Elisabeth & Alexander Bergs
(Guest Eds.) (2017) Toward a multimodal construction grammar (Special Issue). Linguistics Vanguard, 3, s1.
Zlatev, Jordan
(2005) What’s in a schema? Bodily mimesis and the grounding of language. In Beate Hampe & Joe Grady (Eds.), From perception to meaning: Image schemas in cognitive linguistics (pp. 313–342). Berlin & New York: Mouton de Gruyter.
Cited by
Cited by 8 other publications
Fricke, Ellen & Irene Mittelberg
2018. Gesten. In Handbuch Pragmatik, ► pp. 312 ff.
Gawne, Lauren, Chelsea Krajcik, Helene N. Andreassen, Andrea L. Berez-Kroeker & Barbara F. Kelly
2018. Can gestures help clarify the meaning of the Spanish marker ‘se’?. Lingua 208 ► pp. 1 ff.
Meissl, Katharina, Paul Sambre & Kurt Feyaerts
2022. Mapping musical dynamics in space. A qualitative analysis of conductors' movements in orchestra rehearsals. Frontiers in Communication 7
Mittelberg, Irene
2018. Gestures as image schemas and force gestalts: A dynamic systems approach augmented with motion-capture data analyses. Cognitive Semiotics 11:1
Mittelberg, Irene
2019. Visuo-Kinetic Signs Are Inherently Metonymic: How Embodied Metonymy Motivates Forms, Functions, and Schematic Patterns in Gesture. Frontiers in Psychology 10
Mittelberg, Irene
2019. Peirce’s universal categories: On their potential for gesture theory and multimodal analysis. Semiotica 2019:228 ► pp. 193 ff.
Schmitt, Rudolf
2022. Multimodale Metaphernanalyse. Skizze einer szenischen Erweiterung. In Die Praxis der systematischen Metaphernanalyse, ► pp. 217 ff.
This list is based on CrossRef data as of 20 may 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.