In recent years, consensus has been growing that certain co-speech gestures must be incorporated in grammatical theory. Contrary to traditional beliefs, gestural expression has been found to be systematic in various ways, to exhibit some degree of compositional structure and to be to a certain extent language-specific. Most current work on the gesture-grammar interface is, nonetheless, rather programmatic in character or lacks an appropriate theoretical framework. This paper shows that Functional Discourse Grammar (FDG) provides a promising model for understanding the grammar-gesture relationship in a more comprehensive and detailed way than has thus far been achieved. It first evaluates how a FDG perspective on language structure aligns with the ways gestures have been studied. Then it demonstrates that various types of co-speech manual gestures can be classified (perhaps even exhaustively) according to the pragmatic and semantic layers recognized in FDG. On the basis of this survey, a first sketch of a multi-channel (speech-gesture) version of FDG’s architecture is presented. Finally, some challenges are put forward that arise when taking the multimodal grammar enterprise seriously. Altogether, the content of this paper suggests that a multimodal perspective on grammar can be mutually beneficial for functionally oriented linguists and gesture researchers.
1999Mapping the range of information contained in the iconic hand gestures that accompany spontaneous speech. Journal of Language and Social Psychology 18(4). 438–462.
Bergmann, Kirsten & Stefan Kopp
2010Systematicity and idiosyncrasy in iconic gesture use: Empirical analysis and computational modeling. In Stefan Kopp & Ipke Wachsmuth (eds.), Gesture in embodied communication and human-computer interaction, 182–194. Berlin: Springer.
2012Repetitions in gesture: Structures, functions, and cognitive aspects. Frankfurt (Oder): European University Viadrina PhD thesis.
Butler, Christopher S
2008Cognitive adequacy in structural-functional theories of language. Language Sciences 30(1). 1–30.
1990The semiotics of French gestures. Bloomington, IN: Indiana University Press.
1972Some relationships between body motion and speech. In Aaron W. Siegman & Benjamin Pope (eds.), Studies in dyadic communication, 177–210. Elmsford, NY: Pergamon.
1988How gestures can become like words. In Fernando Poyatos (ed.), Cross-cultural perspectives in nonverbal communication, 131–141. Toronto: C. J. Hogrefe.
1992Some recent work from Italy on quotable gestures (Emblems). Journal of Linguistic Anthropology 21. 92–108.
1995Gestures as illocutionary and discourse structure markers in Southern Italian conversation. Journal of Pragmatics 23(3). 247–279.
2004Gesture: Visible action as utterance. Cambridge: CUP.
Kendon, Adam, Thomas A. Sebeok & Jean Umiker-Sebeok
1981Nonverbal communication, interaction, and gesture: Selections from Semiotica. Berlin: de Gruyter.
Kendon, Adam & Laura Versante
2003Pointing by hand in Neapolitan. In Sotaro Kita (ed.), 109–137. Pointing: Where language, culture, and cognition meet. Mahwah, NJ: Erlbaum.
2000How representational gestures help speaking. In David McNeill (ed.), Language and gesture, 162–185. Cambridge: CUP.
2003Pointing: Where language, culture, and cognition meet. New York, NY: Erlbaum.
2009Cross-cultural variation of speech-accompanying gesture: A review. Language and Cognitive Processes 24(2). 145–167.
Kita, Sotaro & Asli Özyürek
2003What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language 48(1). 16–32.
Kress, Gunther R
2000Multimodality: Challenges to thinking about language. Tesol Quarterly 34(2). 337–340.
Kress, Gunther R. & Theo van Leeuwen
1996Reading images: The grammar of visual design. London: Routledge.
2005The relationship between grammar and discourse: Evidence from the Latin particle quidem. Latina Lingua. 576–590.
Ladewig, Silva H
2011Putting the cyclic gesture on a cognitive basis. CogniTextes. Revue de l’Association française de linguistique cognitive 61. Available online at [URL].
Ladewig, Silva H
2012Syntactic and semantic integration of gestures into speech: Structural, cognitive, and conceptual aspects. Frankfurt (Oder): European University Viadrina PhD thesis.
Ladewig, Silva H. & Jana Bressem
2013New insights into the medium hand: Discovering recurrent structures in gestures. Semiotica 1971. 203–231.
Langacker, Ronald W
2001Discourse in cognitive grammar. Cognitive Linguistics 12(2). 143–188.
Langacker, Ronald W
2008Cognitive grammar: A basic introduction. Oxford: OUP.
Levelt, Wim J.M
1989Speaking: From intention to articulation. Cambridge, MA: MIT Press.
Levinson, Stephen C
2003Space in language and cognition: Explorations in cognitive diversity. Cambridge: CUP.
Levy, Elena T. & David McNeill
1992Speech, gesture, and discourse. Discourse Processes 15(3). 277–301.
Lücking, Andy, Kirsten Bergmann, Hahn Florian, Stefan Kopp & Hannes Rieser
2010The Bielefeld speech and gesture alignment corpus (SaGA). Proceedings of the LREC 2010 workshop on Multimodal corpora–advances in capturing, coding and analyzing multimodality.
Mackenzie, J. Lachlan
2010More tiles on the roof: Further thoughts on incremental language production. In Kasper Boye & Elisabeth Engberg-Pedersen (eds.), Language usage and language structure, 263–294. Berlin: Mouton.
1992Hand and mind. What the hands reveal about thought. Chicago: University of Chicago Press.
2005Gesture and thought. Chicago, IL: University of Chicago Press.
McNeill, David & Susan D. Duncan
2000Growth points in thinking-for-speaking. In David McNeill (ed.),
McNeill, David & Elena Levy
1982Conceptual representations in language activity and gesture. In Robert J. Jarvella & Wolfgang Klein (eds.), Speech, place and action: Studies in deixis and related topics. New York, NY: Wiley.
Mittelberg, Irene & Tanja Mortelmans
2013Grammaticalized manual actions of giving and holding: From ditransitive to existential constructions in speech and gesture. Paper presented at the
International Conference of Cognitive Linguistics (ICLC12)
, Edmonton, Canada.
1998Iconicity and gesture. In Serge Santi (ed.), Oralité et Gestualité: Communication Multimodale, Interaction, 321–328. Montréal: L’Harmattan.
2004Forms and uses of the Palm Up Open Hand: A case of a gesture family. In Cornelia Müller & Roland Posner (eds.), The semantics and pragmatics of everyday gestures, 234–256. Berlin: Weidler Verlag.
2013Gestures as a medium of expression: The linguistic potential of gestures. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Sedinha Teßendorf (eds.), 202–217.
Müller, Cornelia, Jana Bressem & Silva Ladewig
2013aTowards a grammar of gestures: A form-based view. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Sedinha Teßendorf (eds.), 707–732.
Müller, Cornelia, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Sedinha Teßendorf
(eds.)2013bBody – Language – Communication: An international handbook on multimodality in human interaction, vol. 11. Berlin: Mouton.
Müller, Cornelia, Silva H. Ladewig & Jana Bressem
2013cGestures and speech from a linguistic perspective: A new field and its history. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Sedinha Teßendorf (eds.), 55–81.
2002A modality-free notion of gesture and how it can help us with the morpheme vs. gesture question in sign language linguistics (or at least give us some criteria to work with). In Richard P. Meier, Kearsy Cormier & David Quinto-Pozos (eds.), Modality and structure in signed and spoken languages, 175–198. Cambridge: CUP.
1998Regular metaphoricity in gesture: Bodily-based models of speech interaction.
Actes du 16e Congrès International des Linguistes
1985Lexicalization patterns: Semantic structure in lexical forms. In Timothy Shopen (ed.), Language typology and syntactic description Vol. 3: Grammatical categories and the lexicon, 57–149. Cambridge: CUP.
Turner, Mark & Francis Steen
2012Multimodal Construction Grammar. In Michael Borkent, Barbara Dancygier & Jennifer A.J. Hinnell (eds.), Language and the creative mind, 255–274. Stanford, CA: CSLI Publications.
Webb, Rebecca A
1996Linguistic features of metaphoric gestures. New York, NY: University of Rochester PhD thesis.
2004Cognitive iconicity: Conceptual spaces, meaning, and gesture in signed language. Cognitive Linguistics 15(2). 119–148.
Wilcox, Sherman & André Nogueira Xavier
2013A framework for unifying spoken language, signed language, and gesture. Revista Todas as Letras 15(1). 88–110.
2003Why pointing with the index finger is not a universal (in sociocultural and semiotic terms). In Sotaro Kita (ed.), 171–215.
2014English multimodal motion constructions: A Construction Grammar perspective.
Papers of the Linguistic Society of Belgium
81. Available online at [URL]
Cited by 6 other publications
2022. Text, discourse, context: A meta-trilogy for discourse analysis. Journal of Pragmatics 199 ► pp. 91 ff.
2022. The Interaction Space. In Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Anthropometry, Human Behavior, and Communication [Lecture Notes in Computer Science, 13319], ► pp. 243 ff.
Lepic, Ryan & Corrine Occhino
2018. A Construction Morphology Approach to Sign Language Analysis. In The Construction of Words [Studies in Morphology, 4], ► pp. 141 ff.
Martin, J. R. & Michele Zappavigna
2019. Embodied meaning: a systemic functional perspective on paralanguage. Functional Linguistics 6:1
2017. Multimodal existential constructions in German: Manual actions of giving as experiential substrate for grammatical and gestural patterns. Linguistics Vanguard 3:s1
This list is based on CrossRef data as of 23 september 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.