Technology has swept our world making it easier to produce and disseminate verbal, visual and aural content on various platforms. This makes communication in the twenty-first century increasingly multimodal. Such a tendency necessitates an inferential approach, rather than an approach based on the code model as in traditional multimodal analysis, wherein “codes and the tacit rules and constraints that underlie the production and interpretation of meaning within each code” are identified (Chandler 2017, 185; also see Kress 2010, 34–36). In pragmatics, text comprehension is often conceptualized as “a context building process” (Maillat 2013), through which contextual assumptions are made salient to help the addressees draw inferences about the text producer’s intended meanings. In multimodal texts, the context building process is rather more complex, as it requires inferring contextual assumptions based on the contents communicated across the different modes. Although such an inferential process is a pragmatic process par excellence, multimodality has tended to fall outside the interest of mainstream pragmatic scholarship. This is because pragmatics, broadly defined as the study of content-sensitive meaning, has traditionally been concerned with the study of verbal communication (Dicerto 2018, 37). Fortunately, the field has recently witnessed an increase in the number of studies applying pragmatic theories to account for the meaning communicated multimodally (e.g. Forceville 2020; Forceville and Clark 2014).

References

Ariel, Mira

2010 Defining Pragmatics. Cambridge: Cambridge University Press.

Arnheim, Rudolph

1969 Art and Visual Perception: A Psychology of the Creative Eye. Berkeley: University of California Press.

Barthes, Roland

1977 Rhetoric of the Image. In Image, Music, Text, transl. by Stephen Heath, 32–51. London: Macmillan.

Bateman, John

2014 Text and Image: A Critical Introduction to the Visual/Verbal Divide. New York: Routledge.

Chandler, Daniel

2017 Semiotics: The Basics. 3rd edition. New York: Routledge.

Clark, Billy

2013 Relevance Theory. Cambridge: Cambridge University Press.

Clark, Billy

2022 Pragmatics the Basics. New York: Routledge.

Cornevin, Vanessa, and Charles Forceville

2017 “From Metaphor to Allegory: The Japanese Manga Afuganisu-tan.” Metaphor and the Social World 7 (2): 235–251.

Crisp, Peter

2005 “Allegory and Symbol — A Fundamental Opposition?” Language and Literature 14: 323–338.

Dicerto, Sara

2018 Multimodal Pragmatics and Translation: A New Model for Source Text Analysis. London: Palgrave Macmillan.

Falkum, Ingrid Lossius

2019 “Metaphor and Metonymy in Acquisition: A Relevance- Theoretic Perspective.” In Relevance, Pragmatics and Interpretation: Essays in Honour of Deirdre Wilson, ed. by Kate Scott, Billy Clark, and Robyn Carston, 205–217. Cambridge: Cambridge University Press.

Fauconnier, Gilles, and Mark Turner

2002 The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities. New York: Basic Books.

Finnegan, A. Cara

2001 “The Naturalistic Enthymeme and Visual Argument: Photographic Representation in the Skull Controversy.” Argumentation and Advocacy 37: 133–149.

Forceville, Charles

2020 Visual and Multimodal Communication: Applying the Relevance Principle. New York: Oxford University Press.

Forceville, Charles, and Billy Clark

2014 “Can Pictures Have Explicatures?” Linguagem em (Dis)curso 14: 451–472.

Garcés-Conejos Blitvich, Pilar

2022 “Moral Emotions, Good Moral Panics, Social Regulation, and Online Public Shaming.” Language & Communication 84: 61–75.

Grice, H. Paul

1975 “Logic and Conversation.” In Syntax and Semantics (Vol. 3): Speech Acts, ed. by Peter Cole, and Jerry L. Morgan, 41–58. New York: Academic Press.

Jewitt, Carey

2013 “Multimodal Methods for Researching Digital Technologies.” In The SAGE Handbook of Digital Technology Research, eds. by Sara Price, Carey Jewitt, and Barry Brown, 240–260. London: SAGE.

Kjeldsen, Jens

2015 “Where Is Visual Argument?” In Reflections on Theoretical Issues in Argumentation Theory, ed. by Frans H. van Eemeren, and Bart Garssen, 107–117. Cham: Springer.

Kress, Gunther

2010 Multimodality: A Social Semiotic Approach to Contemporary Communication. Oxon: Routledge.

Lakoff, George, and Mark Johnson

1980 Metaphors We Live By. Chicago: The University of Chicago Press.

Leech, Geoffrey Neil

1985 “Stylistics.” In Discourse and Literature, ed. by Teun A. Van Dijk, 39–57. John Benjamins.

Lindblom, Kenneth

2009 “Cooperative Principle.” In Concise Encyclopedia of Pragmatics, 2nd edition, ed. by Mey Jacob, 151–158. Oxford: Elsevier.

Machin, David

2016 Introduction to Multimodal Analysis. London: Bloomsbury.

Maillat, Didier

2013 “Constraining Context Selection: On the Pragmatic Inevitability of Manipulation.” Journal of Pragmatics 59: 190–199.

Markham, Annette, and Elizabeth Buchanan

2012 “Ethical Decision-Making and Internet Research: Recommendations from the AoIR Ethics Working Committee (Version 2.0).” https://​aoir​.org​/reports​/ethics2​.pdf

Marlow, Mikaela

2017 “Public Discourse and Intergroup Communication.” In Oxford Research Encyclopedia of Communication, ed. by Jon Nussbaum. Oxford: Oxford University Press.

Martinec, Radan, and Andrew Salway

2005 “A System for Image-Text Relations in New (and Old) Media.” Visual Communication 4 (3): 337–371.

McQuarrie, Edward F., and Barbara J. Phillips

2005 “Indirect Persuasion in Advertising: How Consumers Process Metaphors Presented in Pictures and Words.” Journal of Advertising 34 (2): 7–20.

Moeschler, Jacques

2017 “Formal and Natural Languages: What Logic Tell Us About Natural Language.” In The Routledge Handbook of Pragmatics, ed. by Anne Barron, Yueguo Gu, and Gerard Steen, 241–256, Oxon: Routledge.

O’Halloran, Kay L.

2011 “Multimodal Discourse Analysis.” In The Continuum Companion to Discourse Analysis, ed. by Ken Hyland, and Brian Paltridge, 120–137. London: Continuum.

O’Halloran, Kay L., Sabine Tan, and Marissa K. L. E.

2013 “Multimodal Pragmatics.” In Pragmatics of Discourse, eds. by Klaus P. Schneider, and Anne Barron, 239–268. Berlin: De Gruyter Mouton.

Pilkington, Adrian

2000 Poetic Effects: A Relevance Theory Perspective. Amsterdam: John Benjamins.

Searle, John R.

1979 Expression and Meaning: Studies in the Theory of Speech Acts. Cambridge: Cambridge University Press.

Sperber, Dan and Deirdre Wilson

1987 “Presumptions of Relevance.” Behavioral and Brain Sciences 10 (4): 736–753.

Sperber, Dan, and Deirdre Wilson

1995 Relevance: Communication and Cognition. 2nd edition. Oxford: Blackwell.

Sperber, Dan, and Deirdre Wilson

2006 “Relevance Theory.” In The Handbook of Pragmatics, ed. by Laurence R. Horn, and Gregory Ward, 241–256. Oxford: Blackwell.

Sperber, Dan, and Deirdre Wilson

2008 “A Deflationary Account of Metaphors.” In The Cambridge Handbook of Metaphor and Thought, ed. by Raymond W. Gibbs, 84–105. Cambridge: Cambridge University Press.

Terkourafi, Marina

2021 “Inference and Implicature.” In Cambridge Handbook of Sociopragmatics, ed. by Michael Haugh, Daniel Kadar, and Marina Terkourafi, 30–47. Cambridge: Cambridge University Press.

Unger, Christoph

2019 “Allegory in Relation to Metaphor and Irony.” In Relevance, Pragmatics and Interpretation: Essays in Honour of Deirdre Wilson, ed. by Kate Scott, Billy Clark, and Robyn Carston, 240–252. Cambridge: Cambridge University Press.

Wilson, Deirdre

2013 “Irony Comprehension: A Developmental Perspective.” Journal of Pragmatics 59: 40–56.

Wilson, Deirdre

2016 “Relevance Theory.” In The Oxford Handbook of Pragmatics, ed. by Yang Huang, 1–23. Oxford: Oxford University Press.

Wilson, Deirdre

2017 “Irony, Hyperbole, Jokes and Banter”. In Formal Models in the Study of Language: Applications in Interdisciplinary Contexts, ed. by Joanna Blochowiak, Cristina Grisot, Stephanie Durrleman, and Christopher Laenzlinger, 201–219. Cham: Springer International Publishing.

Wilson, Deirdre

2018 “Relevance Theory and Literary Interpretation.” In Reading Beyond the Code: Literature and Relevance Theory, ed. by Terence Cave, and Deirdre Wilson, 185–204. Oxford: Oxford University Press.

Wilson, Deirdre

2022 “Communication, Comprehension and Interpretation”. In Dynamism in Metaphor and Beyond (Festschrift for Ray Gibbs), ed. by Herbert Colston, Teenie Matlock, and Gerard Steen, 143–155. Amsterdam: John Benjamins.

Wilson, Deirdre, and Dan Sperber

2007 “On Verbal Irony.” In Irony in Language and Thought: A Cognitive Science Reader, ed. by Raymond Gobbs Jr., and Herbert Colston, 35–56. New York: Taylor and Francis.

Wilson, Deirdre, and Dan Sperber

2012 “A Deflationary Account of Metaphor.” In Meaning and Relevance, ed. by Deirdre Wilson, and Dan Sperber, 97–122. Cambridge: Cambridge University Press.

Wilson, Deirdre, and Robyn Carston

2019 “Pragmatics and the Challenge of ‘Non-Propositional’ Effects.” Journal of Pragmatics 145: 31–38.