Cover not available
Part of
Multimodal Communication from a Construction Grammar Perspective
Edited by Kiki Nikiforidou and Mirjam Fried
[Constructional Approaches to Language 38] 2025
► pp. 1–24

Grammar and multimodality

An introduction

ORCID logo | National and Kapodistrian University of Athens
ORCID logo | Charles University
Article outline
  • 1.Common ground
  • 2.Integrating non-verbal features into grammar: Questions and (some) answers
    • 2.1Which non-verbal features to integrate
    • 2.2Non-verbal correlates in the present chapters
  • 3.How to code prosodic and gestural features
  • 4.How to integrate prosodic/phonetic and gesture features into grammar
    • 4.1Sound
    • 4.2Gesture
  • 5.Conclusion and outlook
  • Acknowledgements
  • Notes
  • References
Published online: 16 January 2025
DOI logo
https://doi.org/10.1075/cal.38.intro
References (78)
References
Antonopoulou, E., & Nikiforidou, K. (2011). Construction Grammar and conventional discourse: A construction-based approach to discoursal incongruity. Journal of Pragmatics, 43(10), 2594–2609. DOI logoGoogle Scholar
Βjörkvall, A. (2012). Multimodality. In J. Verschueren, J.-O. Östman, J. Blommaert, & C. Bulcaen (Eds.), Handbook of Pragmatics online volume. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Boas, H. (2013). Cognitive Construction Grammar. In Th. Hoffmann, & G. Trousdale (Eds.), The Oxford Handbook of Construction Grammar (pp. 233–254). Oxford: Oxford University Press. DOI logoGoogle Scholar
Borras-Comes, J., del Mar Vanrell, M., & Prieto, P. (2014). The role of pitch range in establishing intonational contrasts. Journal of International Phonetic Association, 44(1), 1–20. DOI logoGoogle Scholar
Bressem, J., & Ladewig, S. (2011). Rethinking gesture phases: Articulatory features of gestural movement? Semiotica, 184, 53–91. DOI logoGoogle Scholar
Bressem, J., & Müller, C. (2017). The Negative-Assessment-Construction — A multimodal pattern based on a recurrent gesture? Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Cienki, A. (2017). Utterance Construction Grammar (UCxG) and the variable multimodality of constructions. Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Cienki, A. J., & Müller, C. (Eds.). (2008). Metaphor and gesture. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Clark, H. (1973). Space, time, semantics, and the child. In T. E. Moore (Ed.), Cognitive development and the acquisition of language (pp. 27–63). New York: Academic Press. DOI logoGoogle Scholar
Croft, W., & Cruse, D. A. (2004). Cognitive Linguistics. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Debras, C., & Cienki, A. (2012). Some uses of head tilts and shoulder shrugs during human interaction, and their relation to stancetaking. Paper presented at the International Conference on Social Computing. DOI logo
Elvira-García, W. (2019). Two constructions, one syntactic form: Perceptual prosodic differences between elliptical and independent <si + V indicative> clauses in Spanish. In K. Beijering, G. Kaltenböck, & M. S. Sansiñena (Eds.), Insubordination: Theoretical and empirical issues (pp. 240–264). Berlin: De Gruyter Mouton. DOI logoGoogle Scholar
Elvira-García, W., Roseano, P., & Fernández-Planas, A. M. (2017). Prosody as a cue for syntactic dependency. Evidence from dependent and independent clauses with subordination marks in Spanish. Journal of Pragmatics, 109, 29–46. DOI logoGoogle Scholar
Feyaerts, K., Brône, G., & Oben, B. (2017). Multimodality in interaction. In B. Dancygier (Ed.), The Cambridge Handbook of Cognitive Linguistics (pp. 135–156). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Fillmore, C. (1981). Pragmatics and the description of discourse. In P. Cole (Ed.), Radical pragmatics (pp. 143–166). New York: Academic Press.Google Scholar
Fillmore, C. J., Kay, P., & O’Connor, M. C. (1988). Regularity and idiomaticity in grammatical constructions: The Case of Let Alone. Language, 64(3), 501–538. DOI logoGoogle Scholar
Fischer, K. (2010). Beyond the sentence: Constructions, frames and spoken interaction. Constructions and Frames, 2(2), 185–207. DOI logoGoogle Scholar
Fried, M. (2015). Construction Grammar. In A. Alexiadou, & T. Kiss (Eds.), Handbook of Syntax, vol. 4 (pp. 974–1003). Berlin: Mouton de Gruyter. DOI logoGoogle Scholar
Fried, M., & Machač, P. (2022). Intonation as a cue to epistemic stance in one type of insubordinate clauses. Folia Linguistica, 56(1), 183–214. DOI logoGoogle Scholar
Fried, M., & Östman, J-O. (2004). Construction Grammar: a thumbnail sketch. In M. Fried, & J-O. Östman (Eds.), Construction Grammar in a cross-language perspective (pp. 11–86). Amsterdam: John Benjamins. DOI logoGoogle Scholar
(2005). Construction Grammar and spoken language: the case of pragmatic particles. Journal of Pragmatics, 37(11), 1752–1778. DOI logoGoogle Scholar
Goldberg, A. E. (1995). Constructions: A Construction Grammar approach to argument structure. Chicago: University of Chicago Press.Google Scholar
Goldberg, A. (2003). Constructions: A new theoretical approach to language. Trends in Cognitive Sciences 7(5), 219–224. DOI logoGoogle Scholar
Gras, P. (in press). Insubordination at the interaction of discourse, grammar, and prosody. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Gras, P., & Elvira-Garcia, W. (2021). The role of intonation in Construction Grammar: On prosodic constructions. Journal of Pragmatics, 180, 232–247. DOI logoGoogle Scholar
Hilpert, M. (2014). Construction Grammar and its application to English. Edinburgh: Edinburgh University Press. DOI logoGoogle Scholar
Hinnell, J., & Rice, S. (2019). The embodied marking of stance in North American English discourse: Stacked and idiomatic. Presented at the 15th International Cognitive Linguistics Conference, Kwansei Gakuin University, Nishinomiya, Japan.
Janzen, T. (2012). Lexicalization and grammaticalization. In M. Steinbach, R. Pfau, & B. Woll (Eds.), Sign language. An international handbook (pp. 816–840). Berlin: Mouton de Gruyter. DOI logoGoogle Scholar
Jehoul, A., Brône, G., & Feyaerts, K. (2017). The shrug as marker of obviousness. Corpus evidence from Dutch face-to-face conversations. Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Jun, S.-A. (Ed.). (2005). Prosodic typology: The phonology of intonation and phrasing. Oxford: Oxford University Press.Google Scholar
Kay, P., & Fillmore, C. J. (1999). Grammatical constructions and linguistic generalizations: The What’s X doing Y? construction. Language, 75, 1–33. DOI logoGoogle Scholar
Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), Nonverbal communication and language (pp. 207–227). The Hague: Mouton.Google Scholar
(2004). Gesture. Visible action as utterance. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Kress, G., & Van Leeuwen, T. (2006). Reading images: The grammar of visual design. London/ New York: Routledge. DOI logoGoogle Scholar
Ladd, R. (2008). Intonational phonology. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Lakoff, G. (1987). Women, fire and dangerous things. What categories reveal about the mind. Chicago: Chicago University Press. DOI logoGoogle Scholar
Lambrecht, K. (1994). Information structure and sentence form: Topic, focus, and the mental representations of discourse referents. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Lanwer, J. P. (2017). Apposition: A multimodal construction? The multimodality of linguistic constructions in the light of usage-based theory. Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Levinson, S. C. (2006). On the human ‘interaction engine’. In N. J. Enfield, & S. C. Levinson (Eds.), Roots of human sociality: Culture, cognition and interaction (pp. 39–69). Oxford, UK: Berg.Google Scholar
Levinson, S. C., & Holler, J. (2014). The origin of human multi-modal communication. Philological Transactions Royal Society, B, 369: 20130302. DOI logoGoogle Scholar
Liddell, S. K. (2011). Agreement disagreements. Theoretical Linguistics, 37(3–4), 161–172. DOI logoGoogle Scholar
Lindström, J., & Londen, A.-M. (2008). Constructing reasoning: The connectives för att (causal), så att (consecutive) and men att (adversative) in Swedish conversations. In J. Leino (Ed.), Constructional reorganization (pp. 105–152). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Linell, P. (2009). Grammatical constructions in dialogue. In A. Bergs, & G. Diewald (Eds.), Contexts and constructions (pp. 97–110). Amsterdam & Philadelphia: John Benjamins. DOI logoGoogle Scholar
Lorenz, D. (2013). Semi-modal contractions in English: The emancipating effect of frequency. Freiburg: Rombach.Google Scholar
Lücking, A. (2021). Gesture. In S. Müller, A. Abeillé, R. D. Borsley, & J.-P. Koenig (Eds.), Head-Driven Phrase Structure Grammar: The handbook (pp. 1201–1250). Berlin: Language Science Press. DOI logoGoogle Scholar
Lücking, A., Ginzburg, J., & Cooper, R. (2021). Grammar in dialogue. In S. Müller, A. Abeillé, R. D. Borsley, & J.-P. Koenig (Eds.), Head-Driven Phrase Structure Grammar: The handbook (pp. 1155–1199). Berlin: Language Science Press. DOI logoGoogle Scholar
Marandin, J.-M. (2007). Contours as constructions. Constructions Online.Google Scholar
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. DOI logoGoogle Scholar
(2005). Gesture and thought. Chicago: University of Chicago Press. DOI logoGoogle Scholar
Michaelis, L., & Lambrecht, K. (1996). Toward a construction-based theory of language function: The case of nominal extraposition. Language, 72, 215–247. DOI logoGoogle Scholar
Mittelberg, I. (2014). Multimodal existential constructions in German and English. Presentation at GCLA 6, Erlangen, Nurnberg.
(2017). Multimodal existential constructions in German: Manual actions of giving as experiential substrate for grammatical and gestural patterns. Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Müller, C. (2004). Forms and uses of the Palm Up Open Hand: A case of a gesture family? In C. Müller, & R. Posner (Eds.), The semantics and pragmatics of everyday gesture (pp. 233–256). Berlin: Weidler Verlag.Google Scholar
Niebuhr, O. (2015). Stepped intonation contours: A new field of complexity. In R. Skarnitzl, & O. Niebuhr (Eds.), Tackling the complexity of speech (pp. 39–74). Prague: Charles University Press.Google Scholar
(2019). Pitch accents as multiparametric configurations of prosodic features: Evidence from pitch-accent specific micro-rhythms in German. In A. M. Nyvad (Ed.), A sound approach to language matters: In honor of Ocke-Schwen Bohn (pp. 321–351). Aarhus: Aarhus University.Google Scholar
Niebuhr, O., & Ward, N. G. (2018). Challenges in the modeling of pragmatics-related prosody: Introduction to the special issue. Journal of the International Phonetic Association, 48, 1–8. DOI logoGoogle Scholar
Nikiforidou, K. (2021). Grammatical variability and the grammar of genre: Constructions, conventionality, and motivation in ‘stage directions’. Journal of Pragmatics, 173, 189–199. DOI logoGoogle Scholar
Nikiforidou, K., Marmaridou, S., & Mikros, G. K. (2014). What’s in a dialogic construction? A constructional approach to polysemy and the grammar of challenge. Cognitive Linguistics, 25(4), 655–699. DOI logoGoogle Scholar
Ogden, R. (2010). Prosodic constructions in making complaints. In D. Barth-Weingarten, E. Reber, & M. Selting (Eds.), Prosody in interaction (pp. 81–103). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Poldvere, N., & Paradis, C. (2020). ‘What and then a little robot brings it to you?’ The reactive what-x construction in spoken dialogue. English Language and Linguistics, 24(2), 307–332. DOI logoGoogle Scholar
Sag, I. (2012). Sign-Based Construction Grammar: An informal synopsis. In H. Boas, & I. Sag (Eds.), Sign-Based Construction Grammar. Stanford: CSLI Publications.Google Scholar
Schoonjans, S. (2014). Modalpartikeln als multimodale Konstruktionen. Eine korpusbasierte Kookkurrenzanalyse von Modalpartikeln und Gestik im Deutschen. KU Leuven dissertation. Leuven.
(2017). Multimodal Construction Grammar issues are Construction Grammar issues. Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Sommerer, L., & Van de Velde, F. (in press). Constructional networks. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Sweetser, E. (2009). What does it mean to compare language and gesture? Modalities and contrasts. In J. Guo, E. Lieven, & N. Budwig (Eds.), Crosslinguistic approaches to the psychology of language: Studies in the tradition of Dan Isaac Slobin (pp. 357–366). New York: Psychology Press. DOI logoGoogle Scholar
Tabacaru, S., & Lemmens, M. (2014). Raised eyebrows as gestural triggers in humour: The case of sarcasm and hyper-understanding. European Journal of Humour Research, 2(22),11–31. DOI logoGoogle Scholar
Uhrig, P. (2019). Crossmodal collostructions — A usage-based approach to multimodal communication in a Construction Grammar framework. Paper presented at the 52nd Annual Meeting of the Societas Linguistica Europaea, Leipzig.
Ward, N. G. (2019). Prosodic patterns in English conversation. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
(in press). Prosodic constructions. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Wells, J. (2006). English intonation: An introduction. Cambridge: Cambridge University Press.Google Scholar
(2005). Routes from gesture to language. Revista da ABRALIN — Associação Brasileira de Lingüística, 4, 11–45.Google Scholar
Wilcox, S., & Martínez, R. (in press). Constructional approaches to signed language. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Ziem, A. (2017). Do we really need a Multimodal Construction Grammar? Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Zima, E. (2017). On the multimodality of [all the way from X PREP Y]. Linguistics Vanguard Vol. 3, 2016–0055. DOI logoGoogle Scholar
(in press). Construction Grammar and gesture. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Zima, E., & Bergs, A. (2017). Multimodality and construction grammar. Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar