Cover not available
Part of
Multimodal Communication from a Construction Grammar Perspective
Edited by Kiki Nikiforidou and Mirjam Fried
[Constructional Approaches to Language 38] 2025
► pp. 220–250

Towards a functional perspective on multimodal constructions

Evidence from requesting and stance-related Tell me about it

| University of Potsdam
The paper argues in favor of taking a functional perspective on constructions. To this end, a stance-related construction, Tell me about it, and its multimodal delivery in spoken language will be analyzed on a larger scale using the NewsScape Library of International Television News (Steen & Turner, 2013). The quantitative analysis shows that stance-related Tell me about it is longer in duration, accompanied by head movements and raised eyebrows, and the withdrawal of eye contact with the recipient. These findings will eventually be treated as evidence for the view that (multimodal) constructions and some unimodal, nonverbal constructions form a network of functionally related constructions.
Keywords: stance, stance-related constructions, quantitative methods, corpus linguistics, prototype approach
Article outline
  • 1.Introduction
  • 2.Construction grammar and the multimodality of interactions
  • 3.The ambiguity of Tell me about it
  • 4.Multimodal stance-taking
  • 5.Method
    • 5.1A note on the methodological approach
    • 5.2The dataset
    • 5.3Variables and values
    • 5.4Statistical analysis
  • 6.Results
  • 7.Prototypical instances of requesting and stance-related Tell me about it
  • 8.Interim conclusions on multimodal aspects of Tell me about it
  • 9.Why Tell me about it is a multimodal construction (with prototype structure)
    • 9.1Frequencies
    • 9.2High (accumulated) cue validity of features
    • 9.3Why not other features?
    • 9.4Speakers pay attention to features
  • 10.Towards a function-first perspective
  • 11.Summary and conclusions
  • Notes
  • References
This content is being prepared for publication; it may be subject to changes.
References (73)
References
Arppe, A., Gilquin, G., Glynn, D., Hilpert, M., & Zeschel, A. (2010). Cognitive corpus linguistics: Five points of debate on current theory and methodology. Corpora, 5(1), 1–27. DOI logoGoogle Scholar
Auer, P. (1992). Introduction: John Gumperz’ approach to contextualization. In P. Auer, & A. Di Luzio (Eds.), The contextualization of language (pp. 39–53). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. DOI logoGoogle Scholar
Boersma, P., & Weenink, D. (2019). Praat: doing phonetics by computer (Version 6.0. 52 [Windows]). Amsterdam.Google Scholar
Bonaiuto, M., & Bonaiuto, T. (2014). Gestures and body language in Southern Europe: Italy. In M. Cornelia, C. Alan, F. Ellen, L. Silva, M. David, & B. Jana (Eds.), Body — Language — Communication (Vol. 2, pp. 1240–1253). Berlin: De Gruyter Mouton. DOI logoGoogle Scholar
Brône, G., Oben, B., Jehoul, A., Vranjes, J., & Feyaerts, K. (2017). Eye gaze and viewpoint in multimodal interaction management. Cognitive Linguistics, 28(3), 449–483. DOI logoGoogle Scholar
Bryant, G. A. (2010). Prosodic contrasts in ironic speech. Discourse Processes, 47(7), 545–566. DOI logoGoogle Scholar
Bryant, G. A., & Fox Tree, J. E. (2005). Is there an ironic tone of voice? Language and Speech, 48(3), 257–277. DOI logoGoogle Scholar
Caucci, G. M., & Kreuz, R. J. (2012). Social and paralinguistic cues to sarcasm. Humor, 25(1), 1–22. DOI logoGoogle Scholar
Cheang, H. S., & Pell, M. D. (2008). The sound of sarcasm. Speech communication, 50(5), 366–381. DOI logoGoogle Scholar
Cienki, A. (2017). Utterance Construction Grammar (UCxG) and the variable multimodality of constructions. Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Clift, R. (2021). Embodiment in dissent: The eye roll as an interactional practice. Research on Language and Social Interaction, 54(3), 261–276. DOI logoGoogle Scholar
Colston, H. L. (2020). Eye-rolling, irony and embodiment. In A. Athanasiadou, & H. L. Colston (Eds.), The Diversity of irony (pp. 211–235). Berlin/Boston: de Gruyter Mouton. DOI logoGoogle Scholar
Couper-Kuhlen, E. (2016). A sequential approach to affect: The case of ‘disappointment’. In M. Haakana, M. Laakso, & J. Lindström (Eds.), Talk in interaction: Comparative dimensions (pp. 94–123). Helsinki: Studia Fennica Linguistica.Google Scholar
Couper-Kuhlen, E., & Selting, M. (2018). Interactional linguistics: Studying language in social interaction. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
De Saussure, F. (1916). Cours de linguistique générale. Lausanne: Payot.Google Scholar
Debras, C. (2017). The shrug: Forms and meanings of a compound enactment. Gesture, 16(1), 1–34. DOI logoGoogle Scholar
(2021). Multimodal profiles of je (ne) sais pas in spoken French. Journal of Pragmatics, 182, 42–62. DOI logoGoogle Scholar
Debras, C., & Cienki, A. (2012). Some uses of head tilts and shoulder shrugs during human interaction, and their relation to stancetaking. Paper presented at the 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing. DOI logo
Diessel, H. (2019). The grammar network. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Dix, C., & Groß, A. (2021). Raising both eyebrows in interaction [Paper presentation]. International Pragmatics Conference, online. [URL]
Doherty-Sneddon, G., & Phelps, F. G. (2005). Gaze aversion: A response to cognitive or social difficulty? Memory & Cognition, 33(4), 727–733. DOI logoGoogle Scholar
Ekman, P., & Friesen, W. V. (1978). The Facial action coding system: A technique for The measurement of facial movement. San Francisco: Consulting Psychologists Press.Google Scholar
Feyaerts, K., Brône, G., & Oben, B. (2017). Multimodality in interaction. In B. Dancygier (Ed.), The Cambridge Handbook of Cognitive Linguistics (pp. 135–156). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Goldberg, A. E. (2006). Constructions at work. The nature of generalizations in language. New York: Oxford University Press. DOI logoGoogle Scholar
Gries, S. T. (2003). Towards a corpus-based identification of prototypical instances of constructions. Annual Review of Cognitive Linguistics, 1(1), 1–27. DOI logoGoogle Scholar
Gries, S. T., & Stefanowitsch, A. (2004). Extending collostructional analysis: A corpus-based perspective on alternations. International Journal of Corpus Linguistics, 9(1), 97–129. DOI logoGoogle Scholar
Gumperz, J. J. (1992a). Contextualization and understanding. In A. Duranti, & C. Goodwin (Eds.), Rethinking context. Language as an interactive phenomenon (pp. 229–253). Cambridge: Cambridge University Press.Google Scholar
(1992b). Contextualization revisited. In P. Auer, & A. Di Luzio (Eds.), The contextualization of language (pp. 39–53). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Heritage, J. (2012). Epistemics in action: Action formation and territories of knowledge. Research on Language & Social Interaction, 45(1), 1–29. DOI logoGoogle Scholar
Heritage, J., & Raymond, G. (2005). The terms of agreement: Indexing epistemic authority and subordination in talk-in-interaction. Social Psychology Quarterly, 68(1), 15–38. DOI logoGoogle Scholar
Hoffmann, T. (2017). Multimodal constructs-multimodal constructions? The role of constructions in the working memory. Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Hutchby, I., & Wooffitt, R. (2008). Conversation analysis. Malden, MA: Polity Press.Google Scholar
Imo, W. (2007). Der Zwang zur Kategorienbildung: Probleme der Anwendung der Construction Grammar bei der Analyse gesprochener Sprache. Gesprächsforschung-Online-Zeitschrift zur verbalen Interaktion, 8, 22–45.Google Scholar
Imo, W., & Lanwer, J. P. (2020). Prosodie und Konstruktionsgrammatik. Berlin: De Gruyter. DOI logoGoogle Scholar
Langacker, R. W. (2013). Essentials of cognitive linguistics. Oxford: Oxford University Press.Google Scholar
(2017). Entrenchment in cognitive grammar. In H.-J. Schmid (Ed.), Entrenchment and the psychology of language learning: How we reorganize and adapt linguistic knowledge (pp. 39–56). Berlin: De Gruyter Mouton. DOI logoGoogle Scholar
Lanwer, J. P. (2017). Apposition: A multimodal construction? The multimodality of linguistic constructions in the light of usage-based theory. Linguistics Vanguard, 3(s1). DOI logoGoogle Scholar
Lehmann, C. (2023a). Multimodal markers of irony in televised discourse. A corpus-based approach. In L. Brown, I. Hübscher, & A. H. Jucker (Eds.), Multimodal im/politeness. Signed, spoken, written (pp. 251–274). Amsterdam: John Benjamins. DOI logoGoogle Scholar
(2023b). As if that wasn’t enough: English as if clauses as multimodal utterance constructions. English Language & Linguistics, 27(1), 175–202. DOI logoGoogle Scholar
(2024). The prosody of irony is diverse and sometimes construction-specific. In M. Schlechtweg (Ed.), Interfaces of phonetics (pp. 281–308). Berlin: DeGruyter. DOI logoGoogle Scholar
Lerner, G. (2003). Selecting next speaker: The context-sensitive operation of a context-free organization. Language in Society, 32(2), 177–201. DOI logoGoogle Scholar
Lüdecke, D. (2021). sjPlot: Data visualization for statistics in social science (Version R package version 2.8.10). Retrieved from [URL]
Mauchand, M., Vergis, N., & Pell, M. (2018). Ironic tones of voices. Paper presented at the 9th International Conference on Speech Prosody 2018. DOI logo
McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press. DOI logoGoogle Scholar
Mondada, L. (2018). Multiple temporalities of language and body in interaction: Challenges for transcribing multimodality. Research on Language and Social Interaction, 51(1), 85–106. DOI logoGoogle Scholar
Oxford University Press. (n.d.). Tell, v. In Oxford English Dictionary. Retrieved June 29, 2022, from [URL]
Perniss, P. (2018). Why we should study multimodal language. Frontiers in Psychology, 9. DOI logoGoogle Scholar
Pomerantz, A. (1984). Agreeing and disagreeing with assessments: Some features of preferred/dispreferred turn shaped. In J. M. Atkinson, & J. Heritage (Eds.), Structures of social action (pp. 57–101). Cambridge: Cambridge University Press.Google Scholar
R Core Team (2019). R: A language and environment for statistical computing: R foundation for statistical computing. Retrieved from [URL]
Raymond, G., & Heritage, J. (2006). The epistemics of social relations: Owning grandchildren. Language in Society, 35(5), 677–705. DOI logoGoogle Scholar
Rockwell, P. (2000). Lower, slower, louder: Vocal cues of sarcasm. Journal of Psycholinguistic Research, 29(5), 483–495. DOI logoGoogle Scholar
Rossano, F. (2013). Gaze in conversation. In J. Sidnell, & T. Stivers (Eds.), The Handbook of Conversation Analysis (pp. 308–329). Chichester: Wiley-Blackwell. DOI logoGoogle Scholar
Rossi, G. (2012). Bilateral and unilateral requests: The use of imperatives and mi X? interrogatives in Italian. Discourse Processes, 49(5), 426–458. DOI logoGoogle Scholar
Schegloff, E. A. (1996). Turn organization: one intersection of grammar and interaction. In E. Ochs, E. Schegloff, & S. Thompson (Eds.), Interaction and grammar (pp. 52–133). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Schoonjans, S. (2018). Modalpartikeln als multimodale Konstruktionen: Eine korpusbasierte Kookkurrenzanalyse von Modalpartikeln und Gestik im Deutschen. Berlin: De Gruyter. DOI logoGoogle Scholar
Selting, M., Auer, P., Barth-Weingarten, D., Bergmann, J. R., Bergmann, P., Birkner, K., Couper-Kuhlen, E., Deppermann, A., Gilles, P., Günthner, S., Hartung, M., Kern, F., Mertzlufft, C., Meyer, C., Morek, M., Oberzaucher, F., Peters, J., Quasthoff, U., Schütte, W., & Uhmann, S. (2009). Gesprächsanalytisches Transkriptionssystem 2 (GAT 2). Gesprächsforschung: Online-Zeitschrift zur verbalen Interaktion, 10, 353–402.Google Scholar
Steen, F., & Turner, M. B. (2013). Multimodal construction grammar. In M. Borkent, B. Dancygier, & J. Hinnell (Eds.), Language and the Creative Mind (pp. 255–274). Stanford: CSLI Publications. DOI logoGoogle Scholar
Stefanowitsch, A. (2003). The English imperative: A construction-based approach [Unpublished manuscript].Google Scholar
(2013). Collostructional analysis. In T. Hoffmann, & G. Trousdale (Eds.), The Oxford Handbook of Construction Grammar (pp. 290–306). New York: Oxford University Press. DOI logoGoogle Scholar
Tabacaru, S. (2019). A multimodal study of sarcasm in interactional humor. Berlin: de Gruyter Mouton. DOI logoGoogle Scholar
(2020). Faces of sarcasm. Exploring raised eyebrows with sarcasm in French political debates. In A. Athanasiadou, & H. L. Colston (Eds.), The diversity of irony (pp. 256–277). Berlin/Boston: de Gruyter Mouton. DOI logoGoogle Scholar
Tabacaru, S., & Lemmens, M. (2014). Raised eyebrows as gestural triggers in humour: The case of sarcasm and hyper-understanding. The European Journal of Humour Research, 2(2), 11–31. DOI logoGoogle Scholar
Taylor, J. R. (2019). Prototype effects in grammar. In E. Dąbrowska, & D. Divjak (Eds.), Cognitive Linguistics. Key topics (pp. 127–147). Berlin: de Gruyter. DOI logoGoogle Scholar
Uhrig, P. (2018). Multimodal constructions or crossmodal collostructions? Paper presented at the International Conference on Construction Grammar(s), Paris.
(2019). Crossmodal collostructions — A usage-based approach to multimodal communication in a Construction Grammar framework. Paper presented at the 52nd Annual Meeting of the Societas Linguistica Europaea, Leipzig.
(2021). Multimodal corpus linguistics and machine learning Methods for multimodal communication research. Paper presented at the 2021 International Conference on Multimodal Communication, Changsha. [URL]
Vigliocco, G., Perniss, P., & Vinson, D. (2014). Language as a multimodal phenomenon: implications for language learning, processing and evolution. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 369, 20130292. DOI logoGoogle Scholar
Ward, N. G. (2019). Prosodic patterns in English conversation. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., & Sloetjes, H. (2006). ELAN: A professional framework for multimodality research. Paper presented at the 5th international conference on language resources and evaluation (LREC 2006).
Wootton, A. J. (1997). Interaction and the development of mind. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Zima, E. (2017). Multimodal constructional resemblance. In F. Ruiz de Mendoza Ibáñez, A. Luzondo Oyón, & P. Pérez Sobrino (Eds.), Constructing families of constructions: Analytical perspectives and theoretical challenges (pp. 301–337). Amsterdam: John Benjamins. DOI logoGoogle Scholar
(submitted). Multimodale Analysen zu Fremd — und Selbstwahl in ZOOM-Videokonferenzen. In S. Kabatnik, L. Bülow, M.-L. Merten, & R. Mroczynski (Eds.), Pragmatik multimodal. Tübingen: Narr.